You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "aminghadersohi (via GitHub)" <gi...@apache.org> on 2026/09/01 16:32:37 UTC
[PR] feat(mcp): add typed Bullet chart support [superset]
aminghadersohi opened a new pull request, #43770:
URL: https://github.com/apache/superset/pull/43770
### SUMMARY
Adds complete typed MCP support for the frontend ECharts Bullet plugin (`viz_type: bullet`).
- Adds the exact `chart_type: bullet` discriminated-union member, plugin registry entry, schema discovery examples, tool guidance, and recommendation metadata.
- Models the frontend `controlPanel` / `buildQuery` / `transformProps` contract, including SIMPLE, saved, and SQL metrics; grouped row hierarchies; filters/time controls; ordering/limits; and range, marker, label, legend, and number-format controls.
- Adds dataset-aware exact/casefold normalization, ambiguity detection, role/output validation, and numeric-output constraints.
- Uses the shared MCP query builder for Bullet's singular metric + groupby transformation and preserves frontend-only `orderby` semantics for compile/data paths.
- Preserves omitted native Bullet query, filter, and presentation state across `update_chart` and `update_chart_preview`, while explicit empty/false/null values remain authoritative.
- Adds layered horizontal ASCII/Vega-Lite preview support for saved and unsaved Bullet charts, including metric aliases, the complete dimension hierarchy, ranges, targets, and marker lines.
- Adds product-path coverage through `generate_chart`, `update_chart`, `update_chart_preview`, compile/query builders, registry/schema discovery, dataset normalization, and preview paths; includes a non-Bullet regression guard.
Tracking: [Shortcut SC-119157](https://app.shortcut.com/preset/story/119157)
Agor worktree: [sc-119157-mcp-bullet](https://agor.sandbox.preset.zone/ui/w/01a05dad4b317188b2ac217d/)
Deduplication/coordination: current master, open/merged Apache Superset PRs, contributor/community branches, and Agor review-zone work were checked before implementation. No existing Bullet MCP implementation was found. I also reviewed the newer chart-type MCP work (including #43737 and the community chart-type PR series #43567-#43573) to avoid overlapping implementations.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A — this changes the MCP server contract/query/preview paths and does not modify the frontend Bullet plugin UI. A live Superset server was not available in this worktree, so no UI screenshot was captured.
### TESTING INSTRUCTIONS
Run in a SQLAlchemy 2-compatible environment:
```bash
export PYTHONPATH="$PWD/superset-core/src"
# Bullet product paths
python -m pytest -q tests/unit_tests/mcp_service/chart/test_bullet_chart.py
# Full MCP chart suite
python -m pytest -q tests/unit_tests/mcp_service/chart/
# Broader shared query-context coverage
python -m pytest -q \
tests/unit_tests/common/test_form_data_query_context.py \
tests/unit_tests/common/test_query_context_factory.py \
tests/unit_tests/charts/data/form_data_test.py \
tests/unit_tests/charts/data/test_empty_query_context.py
# Staged/range static checks
pre-commit run --from-ref origin/master --to-ref HEAD
```
Results on this branch:
- Bullet product paths: `23 passed`
- Focused MCP schema/registry/query/tool paths: `560 passed`
- Full MCP chart suite: `1389 passed`
- Broader form-data/query-context paths: `87 passed`
- Pre-commit, mypy, ruff/format, pylint, and repository hooks: passed
Manual schema check: call `get_chart_type_schema` with `chart_type="bullet"`, then use its example with `generate_chart`; verify returned native form data has `viz_type="bullet"`, one `metric`, the complete `groupby`, and comma-separated range/marker controls.
### ADDITIONAL INFORMATION
- [x] Has associated issue: [Shortcut SC-119157](https://app.shortcut.com/preset/story/119157)
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [x] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4174947212
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2538,6 +2539,89 @@ def _metric_display_label(col: ColumnRef) -> str:
return col.label or col.name or ""
+def _require_unique_bullet_order_match(
+ matches: list[tuple[str, int | None]], target: str, role: str
+) -> tuple[str, int | None] | None:
+ """Return a unique sort-role match or reject ambiguity."""
+ if len(matches) == 1:
+ return matches[0]
+ if len(matches) > 1:
+ raise ValueError(f"ambiguous {role}: {target}")
+ return None
+
+
+def _bullet_metric_output_label(metric: ColumnRef) -> str:
+ """Return the field name emitted by Bullet's native metric shape."""
+ if metric.saved_metric:
+ # Saved metrics are serialized as a bare metric-name string; a
+ # ColumnRef display label does not change the query output alias.
+ return metric.name or ""
+ return _metric_display_label(metric)
Review Comment:
For `metric: {"name": "Revenue"}`, this validates the output as `Revenue`, but the mapper emits `SUM(Revenue)`, so grouping by `Revenue` is rejected as a collision and ordering by the actual metric output is rejected as unknown; `STDDEV` similarly maps to `STDDEV_SAMP`. Could output-role validation use the same defaulting and aggregate aliases as `create_metric_object`?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1750,544 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
+ (form_data.get("all_columns") or [])
+ if raw_mode
+ else (form_data.get("groupby") or [])
+ )
+ table_metrics = [] if raw_mode else list(form_data.get("metrics") or [])
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ query["columns"] = [
Review Comment:
With Table grouping on `created_at` and `updated_at` and a daily grain, this wraps both temporal columns, whereas the frontend buckets only the first eligible column. Rows with different `updated_at` hours on the same day are therefore collapsed into one sum only in MCP; could this mirror the frontend's single temporal-axis selection?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,459 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+ if not data:
+ # The strict model's zero-valued ungrouped row is a render-only
+ # frontend fallback, not source query data, so an empty query
+ # exposes no rows to get-data and exports.
+ return [], None
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the dimensions. Its float measure is
+ # render-only: exported rows keep the validated source metric so exact
+ # BIGINT/Decimal values are not rounded to binary64.
+ rows: list[Any] = []
+ for source, row in zip(data, model.rows, strict=False):
+ exposed = dict(row)
+ if model.metric_field in source:
+ exposed[model.metric_field] = _safe_enum_backing(
+ source[model.metric_field]
+ )
+ rows.append(exposed)
+ return rows, None
+
+ def ascii_preview(
+ self, data: list[Any], form_data: dict[str, Any], width: int
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def vega_lite_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> VegaLitePreview | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_bullet_vega_lite_preview,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_bullet_vega_lite_preview(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def resolve_update_config(
+ self,
+ config: Any,
+ existing_form_data: dict[str, Any],
+ *,
+ dataset_rebind: bool,
+ ) -> Any:
+ """Resolve sort targets against the saved hierarchy before mapping."""
+ if not isinstance(config, BulletChartConfig) or config.dimensions is not None:
+ return config
+ if not config.order_by:
+ return config
+ dimensions = (
+ existing_form_data.get("groupby") or []
+ if not dataset_rebind and existing_form_data.get("viz_type") == "bullet"
+ else []
+ )
+ return BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": dimensions}
+ )
+
+ def merge_update_form_data(
+ self,
+ existing_form_data: dict[str, Any],
+ new_form_data: dict[str, Any],
+ config: Any,
+ *,
+ dataset_rebind: bool,
+ ) -> dict[str, Any] | None:
+ """Merge filter provenance and preserve omitted native Bullet controls."""
+ if not isinstance(config, BulletChartConfig) or dataset_rebind:
+ return None
+ from superset.mcp_service.chart.chart_utils import (
+ merge_bullet_form_data,
+ merge_update_form_data,
+ )
+
+ merged = dict(new_form_data)
+ merge_update_form_data(existing_form_data, merged, config)
+ merge_bullet_form_data(existing_form_data, merged)
Review Comment:
A presentation-only update of a saved Bullet drops `url_params` here because this merge starts from mapped controls and skips generic same-viz preservation. A saved SQL metric using `url_param("region", "US")` with chart metadata selecting `EU` can therefore silently query `US` after changing labels; could omitted query metadata survive same-dataset updates?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1781,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
Review Comment:
The new pruning still retains a sort on `SavedRevenue` when the update replaces it with `SUM(Revenue)`, so merged validation rejects the dimension-clear request (the new success test mocks that validation away); case-folding also leaves a stale sort when only one of `Region` and `region` is removed. Could pruning use the final exact output roles, with merged validation enabled in these regressions?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1091,350 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = float(token)
Review Comment:
Native Bullet controls use JavaScript `Number`, so a saved marker `"0x64"` renders at 100 in Explore but is dropped here, while `"1_000"` is ignored in Explore but becomes 1,000 here. Could ranges and marker parsing preserve the frontend tokenizer semantics?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1091,350 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = float(token)
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ category_field = _unique_bullet_category_field(model.rows)
+ row_field = _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_row", (category_field,)
+ )
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field, row_field)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[row_field] = row_index
+ copied[category_field] = (
+ ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ if model.dimensions
+ else ""
+ )
+ if (
+ range_tooltip_field is not None
+ and containing_range_labels[row_index] is not None
+ ):
+ copied[range_tooltip_field] = containing_range_labels[row_index]
+ values.append(copied)
+
+ # Explore uses indexed rows even when their display labels are identical.
+ category_labels = [row[category_field] for row in values]
+ vega_format = {
+ "SMART_NUMBER": "~s",
+ "SMART_NUMBER_SIGNED": "+~s",
+ }.get(model.y_axis_format, model.y_axis_format)
+ y_encoding = {
+ "field": row_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ "sort": None,
+ "axis": {"labelExpr": f"{json.dumps(category_labels)}[datum.value]"},
+ }
+ tooltip = [
+ {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ },
+ {
+ "field": model.metric_field,
+ "type": "quantitative",
+ "format": vega_format,
+ },
+ ]
+ if range_tooltip_field is not None:
+ tooltip.append(
+ {"field": range_tooltip_field, "type": "nominal", "title": "Range"}
+ )
+ axis_min = min(
+ 0.0,
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ axis_max = max(
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ if axis_min == axis_max:
+ axis_max = axis_min + (abs(axis_min) or 1)
+
+ def label_at(labels: list[str], index: int, value: float, prefix: str) -> str:
+ if index < len(labels) and labels[index]:
+ return labels[index]
+ return (
+ ""
+ if prefix == "Range"
+ else _format_bullet_number(model.y_axis_format, value)
+ )
+
+ def legend_color(name: str) -> dict[str, Any]:
+ return {
+ "datum": name,
+ "type": "nominal",
+ "legend": {"title": None} if model.show_legend else None,
+ }
+
+ layers: list[dict[str, Any]] = []
+ range_entries = sorted(
+ [
+ (
+ threshold,
+ label_at(model.range_labels, index, threshold, "Range"),
+ )
+ for index, threshold in enumerate(model.ranges)
+ ],
+ key=lambda entry: entry[0],
+ reverse=True,
+ )
+ for index, (threshold, label) in enumerate(range_entries):
+ layers.append(
+ {
+ "mark": {
+ "type": "rect",
+ "opacity": max(0.08, 0.28 - index * 0.04),
+ },
+ "encoding": {
+ "x": {"datum": axis_min, "type": "quantitative"},
+ "x2": {"datum": threshold},
+ "y": y_encoding,
+ "color": legend_color(
+ (f"{label}: " if label else "")
+ + f"≤ {_format_bullet_number(model.y_axis_format, threshold)}"
+ ),
+ "tooltip": [
+ {"value": label, "title": "Range"},
+ {
+ "value": _format_bullet_number(
+ model.y_axis_format, threshold
+ ),
+ "title": "Threshold",
+ },
+ ],
+ },
+ }
+ )
+ if model.show_labels and label:
+ layers.append(
+ {
+ "mark": {"type": "text", "align": "right", "dx": -3},
+ "encoding": {
+ "x": {"datum": threshold, "type": "quantitative"},
+ "y": y_encoding,
+ "text": {"value": label},
+ },
+ }
+ )
+ layers.append(
+ {
+ "mark": {"type": "bar", "tooltip": True, "size": 16},
+ "encoding": {
+ "x": {
+ "field": model.metric_field,
Review Comment:
A metric labeled `Revenue.total` keeps that literal key in the returned rows, but this Vega field reference treats the dot as a nested lookup, so the measure and tooltip disappear even though Explore renders them. Could metric field references escape dots/brackets or use a safe derived alias?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,455 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
+ ):
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
+
+
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
+ x_axis = form_data.get("x_axis")
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
+ },
+ }
+ )
+ rolling_type = form_data.get("rolling_type")
+ rolling_columns = (
+ [*offset_map.values(), *offset_map.keys()] if comparison else metric_labels
+ )
+ if rolling_type == "cumsum":
+ chain.append(
+ {
+ "operation": "cum",
+ "options": {
+ "operator": "sum",
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ elif rolling_type in {"sum", "mean", "std"}:
+ chain.append(
+ {
+ "operation": "rolling",
+ "options": {
+ "rolling_type": rolling_type,
+ "window": int(form_data.get("rolling_periods") or 1),
+ "min_periods": int(form_data.get("min_periods") or 0),
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ comparison_type = form_data.get("comparison_type")
+ if comparison and comparison_type != "values":
+ chain.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": list(offset_map.values()),
+ "compare_columns": list(offset_map.keys()),
+ "compare_type": comparison_type,
+ "drop_original_columns": True,
+ },
+ }
+ )
+ if complete_timeseries_contract and form_data.get("contributionMode"):
+ chain.append(
+ {
+ "operation": "contribution",
+ "options": {
+ "orientation": form_data["contributionMode"],
+ "time_shifts": offsets if comparison else [],
+ },
+ }
+ )
+ if comparison:
+ rename: dict[str, str | None] = {}
+ for shifted, metric in offset_map.items():
+ offset = next(
+ (item for item in offsets if shifted.endswith(f"__{item}")), None
+ )
+ source = (
+ shifted
+ if comparison_type == "values"
+ else f"{comparison_type}__{metric}__{shifted}"
+ )
+ rename[source] = f"{metric}, {offset}" if len(metrics) > 1 else offset
+ if rename:
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {"columns": rename, "level": 0, "inplace": True},
+ }
+ )
+ elif (
+ x_label
+ and len(metrics) == 1
+ and (series_labels or len(offsets) > 1)
+ and form_data.get("truncate_metric") is not None
+ and form_data.get("truncate_metric")
+ ):
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {
+ "columns": {metric_labels[0]: None},
+ "level": 0,
+ "inplace": True,
+ },
+ }
+ )
+ if complete_timeseries_contract:
+ x_axis_sort = form_data.get("x_axis_sort")
+ x_axis_sort_asc = form_data.get("x_axis_sort_asc")
+ sortable_labels = [
+ label
+ for label in [
+ x_label,
+ *(
+ _metric_label(metric)
+ for metric in _as_list(form_data.get("metrics"))
+ ),
+ *(
+ _metric_label(metric)
+ for metric in _timeseries_extra_metrics(form_data)
+ ),
+ ]
+ if label
+ ]
if (
- not isinstance(entry, (list, tuple))
- or len(entry) != 2
- or not isinstance(entry[0], str)
- or not entry[0]
- or not isinstance(entry[1], bool)
+ x_axis_sort is not None
+ and x_axis_sort_asc is not None
+ and x_axis_sort in sortable_labels
+ and not _as_list(form_data.get("groupby"))
):
- raise ValueError(
- f"Gantt order_by_cols[{index}] must be [column, ascending_boolean]"
- )
- orderby.append([entry[0], entry[1]])
+ options: dict[str, Any] = {"ascending": x_axis_sort_asc}
+ if x_axis_sort == x_label:
+ options["is_sort_index"] = True
+ else:
+ options["by"] = x_axis_sort
+ chain.append({"operation": "sort", "options": options})
+ chain.append({"operation": "flatten"})
+
+ if complete_timeseries_contract and form_data.get("forecastEnabled") and x_label:
+ x_axis_grain = (
+ x_axis.get("timeGrain")
+ if isinstance(x_axis, dict)
+ and x_axis.get("expressionType") in {"SIMPLE", "SQL"}
+ else None
+ )
+ time_grain = (
+ x_axis_grain
+ or (query.get("extras") or {}).get("time_grain_sqla")
+ or form_data.get("time_grain_sqla")
+ or "P1D"
+ )
+ chain.append(
+ {
+ "operation": "prophet",
+ "options": {
+ "time_grain": time_grain,
+ "periods": int(form_data.get("forecastPeriods", 10)),
+ "confidence_interval": float(
+ form_data.get("forecastInterval", 0.8)
+ ),
+ "yearly_seasonality": form_data.get("forecastSeasonalityYearly"),
+ "weekly_seasonality": form_data.get("forecastSeasonalityWeekly"),
+ "daily_seasonality": form_data.get("forecastSeasonalityDaily"),
+ "index": x_label,
+ },
+ }
+ )
+ return [operator for operator in chain if operator is not None]
- columns: list[Any] = []
- seen: set[str] = set()
- for column in (
- start_time,
- end_time,
- category,
- *series_columns,
- *tooltip_columns,
- *(entry[0] for entry in orderby),
+
+def _mixed_layer_form_data(
+ form_data: dict[str, Any], *, secondary: bool
+) -> dict[str, Any]:
+ """Mirror MixedTimeseries remove/retainFormDataSuffix for one layer."""
+ if not secondary:
+ return {
+ key: value for key, value in form_data.items() if not key.endswith("_b")
+ }
+ layer = {key: value for key, value in form_data.items() if not key.endswith("_b")}
+ for isolated_key in (
+ "metrics",
+ "groupby",
+ "orderby",
+ "limit",
+ "series_limit",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ "order_desc",
+ "row_limit",
+ "truncate_metric",
+ "time_compare",
+ "comparison_type",
+ "resample_method",
+ "resample_rule",
+ "rolling_type",
+ "rolling_periods",
+ "min_periods",
+ "show_empty_columns",
):
- key = utils_json.dumps(column, sort_keys=True, default=str)
- if key not in seen:
- seen.add(key)
- columns.append(column)
- return columns, list(raw_tooltip_metrics), orderby, series_columns
+ if f"{isolated_key}_b" not in form_data:
Review Comment:
For a Mixed chart with shared `row_limit: 25` and no `row_limit_b`, this removes the limit from query B, while the frontend retains the shared limit when there is no suffixed override. `get_chart_sql` can therefore describe a differently bounded secondary query from Explore; could shared limits survive this layer isolation?
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,431 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
+ viz_type = form_data.get("viz_type")
+ if strict_all_form_refs and (
+ viz_type == "mixed_timeseries"
+ or (
+ isinstance(viz_type, str)
+ and (
+ viz_type.startswith("echarts_timeseries") or viz_type == "echarts_area"
+ )
+ )
+ ):
+ if (raw_x_axis := form_data.get("x_axis")) is not None and (
+ error := column_error(raw_x_axis, "form-data x_axis column")
+ ):
+ return error
+ metric_fields = [
+ "metrics",
+ "size",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ ]
+ if viz_type == "mixed_timeseries":
+ metric_fields.extend(
+ [
+ "metrics_b",
+ "size_b",
+ "timeseries_limit_metric_b",
+ "series_limit_metric_b",
+ ]
+ )
+ for field_name in metric_fields:
+ raw_value = form_data.get(field_name)
+ values = raw_value if isinstance(raw_value, list) else [raw_value]
+ for value in values:
+ if value is not None and (
+ error := metric_error(value, f"form-data {field_name} metric")
+ ):
+ return error
+
+ layer_suffixes = ("", "_b") if viz_type == "mixed_timeseries" else ("",)
+ for suffix in layer_suffixes:
+ sort_field = f"x_axis_sort{suffix}"
+ if sort_field not in form_data or form_data.get(sort_field) is None:
+ continue
+ x_axis = form_data.get(f"x_axis{suffix}", form_data.get("x_axis"))
+ allowed_labels: set[str] = set()
+ if x_axis_label := _native_column_label(x_axis):
+ allowed_labels.add(x_axis_label)
+ raw_metrics = form_data.get(f"metrics{suffix}")
+ for metric in raw_metrics if isinstance(raw_metrics, list) else []:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ raw_limit_metric = form_data.get(f"timeseries_limit_metric{suffix}")
+ limit_metrics = (
+ raw_limit_metric
+ if isinstance(raw_limit_metric, list)
+ else [raw_limit_metric]
+ )
+ for metric in limit_metrics:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ sort_value = form_data[sort_field]
+ if not isinstance(sort_value, str) or sort_value not in allowed_labels:
+ return _native_validation_error(sort_field, repr(sort_value)[:200])
+
+ if (
+ strict_all_form_refs
+ and isinstance(viz_type, str)
+ and viz_type.startswith("deck_")
+ ):
+ # Deck layers store most query roles outside common columns/metrics.
+ # Validate every renderer-consumed raw control as well as the generated
+ # QueryObject so a dataset-only rebind cannot hide or discard a stale
+ # tooltip, cross-filter, spatial, path, or metric reference.
+ for spatial_field in ("spatial", "start_spatial", "end_spatial"):
+ spatial = form_data.get(spatial_field)
+ if spatial is None:
+ continue
+ if not isinstance(spatial, dict):
+ return _native_validation_error(
+ f"form-data {spatial_field}", repr(spatial)[:200]
+ )
+ spatial_type = spatial.get("type")
+ if not isinstance(spatial_type, str):
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ role_fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if role_fields is None:
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ for role_field in role_fields:
+ spatial_value = spatial.get(role_field)
+ if spatial_value is None:
+ return _native_validation_error(
+ f"form-data {spatial_field}.{role_field} column", "missing"
+ )
+ if error := column_error(
+ spatial_value, f"form-data {spatial_field}.{role_field} column"
+ ):
+ return error
+
+ for field_name in (
+ "line_column",
+ "geojson",
+ "dimension",
+ "cross_filter_column",
+ ):
+ column_value = form_data.get(field_name)
+ if column_value is not None and (
+ error := column_error(column_value, f"form-data {field_name} column")
+ ):
+ return error
+
+ tooltip_contents = form_data.get("tooltip_contents")
+ if tooltip_contents is not None and not isinstance(tooltip_contents, list):
+ return _native_validation_error(
+ "form-data tooltip_contents", repr(tooltip_contents)[:200]
+ )
+ for index, item in enumerate(tooltip_contents or []):
+ tooltip_value: Any = None
+ if isinstance(item, str):
+ tooltip_value = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ tooltip_value = item.get("column_name")
+ if tooltip_value is not None and (
+ error := column_error(
+ tooltip_value, f"form-data tooltip_contents[{index}] column"
+ )
+ ):
+ return error
+
+ metric_values: list[tuple[str, Any]] = []
+ if viz_type not in {"deck_geojson", "deck_polygon"}:
+ for field_name in ("metrics", "metric", "size"):
+ raw_deck_metrics = form_data.get(field_name)
+ deck_metrics = (
+ raw_deck_metrics
+ if isinstance(raw_deck_metrics, list)
+ else [raw_deck_metrics]
+ )
+ metric_values.extend(
+ (f"form-data {field_name} metric", deck_metric)
+ for deck_metric in deck_metrics
+ if deck_metric is not None
+ )
+ if viz_type == "deck_polygon" and form_data.get("metric") is not None:
+ metric_values.append(("form-data metric metric", form_data.get("metric")))
+ fixed_metric_fields = (
+ ("point_radius_fixed",)
+ if viz_type in {"deck_scatter", "deck_polygon"}
+ else ()
+ ) + (("line_width",) if viz_type == "deck_path" else ())
+ for field_name in fixed_metric_fields:
+ fixed_value = form_data.get(field_name)
+ deck_metric: Any = (
+ fixed_value
+ if (
+ isinstance(fixed_value, str)
+ and fixed_value
+ and viz_type != "deck_polygon"
+ )
+ else None
+ )
+ if isinstance(fixed_value, dict) and fixed_value.get("type") == "metric":
+ deck_metric = fixed_value.get("value")
+ if deck_metric is not None:
+ metric_values.append((f"form-data {field_name} metric", deck_metric))
+ if viz_type == "deck_path" and form_data.get("breakpoint_metric") is not None:
+ metric_values.append(
+ (
+ "form-data breakpoint_metric metric",
+ form_data.get("breakpoint_metric"),
+ )
+ )
+ for role, deck_metric in metric_values:
+ if error := metric_error(deck_metric, role):
+ return error
+
+ for filter_ in form_data.get("adhoc_filters") or []:
+ if not isinstance(filter_, dict) or filter_.get("expressionType") != "SIMPLE":
+ continue
+ if not strict_all_form_refs and _is_inert_adhoc_filter(filter_):
+ continue
+ subject = filter_.get("subject")
+ clause = str(filter_.get("clause") or "WHERE").upper()
+ if clause == "HAVING" and isinstance(subject, str):
+ metric_matches = [
+ name for name in saved_metrics if name.casefold() == subject.casefold()
+ ]
+ if len(metric_matches) == 1:
+ continue
+ if subject is not None and (
+ error := column_error(subject, "form-data filter column")
+ ):
+ return error
+ if filter_.get("operator") == "TEMPORAL_RANGE" and isinstance(subject, str):
+ try:
+ temporal = resolve_dataset_column(subject, dataset_context)
+ except ValueError:
+ temporal = None
+ if temporal is not None and not temporal.get("is_temporal", False):
+ return _native_validation_error("temporal filter column", subject)
+
+ temporal_lookup = form_data.get("temporal_columns_lookup")
+ if isinstance(temporal_lookup, dict):
+ for column, enabled in temporal_lookup.items():
+ if enabled and (error := column_error(column, "temporal lookup column")):
Review Comment:
`temporal_columns_lookup` records all temporal columns in the old datasource, including ones the chart never selects. A Table rebind with valid replacement roles still fails here if the new dataset lacks one of those unused dates, because pruning and same-viz merging retain the lookup; could this validate only selected roles or rebuild that metadata for the replacement dataset?
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1338,6 +1340,51 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates; omitted label controls preserve the saved state.
+
+Saved native `ranges`, `markers`, and `marker_lines` controls ignore empty,
+non-numeric, and NaN tokens, matching Explore. If no numeric range remains, the
+preview uses the default band up to 110% of the largest measure. Unrelated
+updates preserve these saved controls. Newly authored typed lists must contain
+finite numbers; infinite native values and oversized controls remain errors.
+
+On same-dataset chart updates, `filters: []` clears both user filters and the
+generated dashboard-time binding. `temporal_column: null` clears only that
+generated binding, preserving user filters. Omitting those controls preserves
+the saved binding. Bullet Vega previews keep separate indexed rows even when
+dimension display labels are identical, and support both `SMART_NUMBER` and
+`SMART_NUMBER_SIGNED` number formats.
+
+### Chart query result limits
+
+MCP chart tools accept up to 50,000 rows per query. The row-shaped `indexnames`
Review Comment:
A one-row query containing a 65,537-byte text cell now fails with `MalformedQueryResult` regardless of the configured response-size guard, but this limits section documents neither the new 64 KiB cell cap nor the 16 MiB aggregate cap. Could the guide and upgrade notes describe these hard limits and the affected query/export paths?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1875 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
return None
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
Review Comment:
A trusted `np.longdouble("1.000000000000000001")` survives DataFrame materialization unchanged but becomes `1.0` here before chart, dataset, and table responses or exports, silently losing precision outside the renderer. Could extended-precision values keep an exact wire representation instead of narrowing to binary64?
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,427 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ matches = [
+ item
+ for item in saved_metrics
+ if item == name or item.casefold() == name.casefold()
+ ]
+ if len(set(matches)) != 1:
Review Comment:
The exact-match fix handles metric roles, but the HAVING branch still gathers all case-folded matches: an exact `HAVING Revenue` falls through to physical-column validation when saved metrics include both `Revenue` and `revenue`, so the valid chart update is still rejected. Could that branch prefer exact saved-metric names too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5968811071
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4226798374
##########
superset/mcp_service/semantic_layer/tool/get_table.py:
##########
@@ -366,23 +372,24 @@ def _build_response(
is_builtin: bool,
display_name: str,
query_result: dict[str, Any],
+ data: list[dict[str, Any]],
+ raw_columns: list[str],
+ coltypes: list[int | GenericDataType],
query_duration_ms: int,
warnings: list[str],
temporal_columns: set[str] | None = None,
valid_grains: dict[str, dict[str, str]] | None = None,
) -> GetTableResponse:
"""Format the query result into a GetTableResponse."""
- data = query_result.get("data", [])
- raw_columns = query_result.get("colnames", [])
cache_status = get_cache_status_from_result(
query_result, force_refresh=request.force_refresh
)
-
if not data:
+ columns_meta = format_data_columns(data, raw_columns, coltypes)
Review Comment:
The empty-result branch builds column metadata without the temporal overrides the non-empty path applies below. For a view with temporal date at monthly grain, get_table with dimensions=[date], time_grain=P1M and a filter matching no rows can report the date__Month column as data_type string, while the same selection with rows reports temporal, so the schema depends on row count. Should result_temporal_columns be computed before the empty branch and passed to format_data_columns here too?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +997,443 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
+
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ offsets: list[Any] = []
+ if _time_comparison(form_data, query.get("metrics") or []):
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
- if (
- not isinstance(entry, (list, tuple))
- or len(entry) != 2
- or not isinstance(entry[0], str)
- or not entry[0]
- or not isinstance(entry[1], bool)
- ):
- raise ValueError(
- f"Gantt order_by_cols[{index}] must be [column, ascending_boolean]"
- )
- orderby.append([entry[0], entry[1]])
- columns: list[Any] = []
- seen: set[str] = set()
- for column in (
- start_time,
- end_time,
- category,
- *series_columns,
- *tooltip_columns,
- *(entry[0] for entry in orderby),
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ **(
+ {"isColumnReference": True}
+ if str(form_data.get("datasource", "")).endswith("__semantic_view")
+ else {}
+ ),
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
):
- key = utils_json.dumps(column, sort_keys=True, default=str)
- if key not in seen:
- seen.add(key)
- columns.append(column)
- return columns, list(raw_tooltip_metrics), orderby, series_columns
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
-def extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
- """Return the x_axis column name from form_data, or None if not set."""
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
x_axis = form_data.get("x_axis")
- if isinstance(x_axis, str) and x_axis:
- return x_axis
- if isinstance(x_axis, dict):
- col_name = x_axis.get("column_name")
- return col_name if isinstance(col_name, str) and col_name else None
- return None
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
+ **(
+ {"fill_time_range": True}
+ if form_data.get("resample_fill_time_range")
+ else {}
+ ),
+ },
+ }
+ )
+ rolling_type = form_data.get("rolling_type")
+ rolling_columns = (
+ [*offset_map.values(), *offset_map.keys()] if comparison else metric_labels
+ )
+ if rolling_type == "cumsum":
+ chain.append(
+ {
+ "operation": "cum",
+ "options": {
+ "operator": "sum",
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ elif rolling_type in {"sum", "mean", "std"}:
+ chain.append(
+ {
+ "operation": "rolling",
+ "options": {
+ "rolling_type": rolling_type,
+ "window": int(form_data.get("rolling_periods") or 1),
+ "min_periods": int(form_data.get("min_periods") or 0),
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ comparison_type = form_data.get("comparison_type")
+ if comparison and comparison_type != "values":
+ chain.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": list(offset_map.values()),
+ "compare_columns": list(offset_map.keys()),
+ "compare_type": comparison_type,
+ "drop_original_columns": True,
+ },
+ }
+ )
+ if complete_timeseries_contract and form_data.get("contributionMode"):
+ chain.append(
+ {
+ "operation": "contribution",
+ "options": {
+ "orientation": form_data["contributionMode"],
+ "time_shifts": offsets if comparison else [],
+ },
+ }
+ )
+ if comparison:
+ rename: dict[str, str | None] = {}
+ for shifted, metric in offset_map.items():
+ offset = next(
+ (item for item in offsets if shifted.endswith(f"__{item}")), None
+ )
+ source = (
+ shifted
+ if comparison_type == "values"
+ else f"{comparison_type}__{metric}__{shifted}"
+ )
+ rename[source] = f"{metric}, {offset}" if len(metrics) > 1 else offset
+ if rename:
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {"columns": rename, "level": 0, "inplace": True},
+ }
+ )
+ elif (
+ comparison_type not in {"difference", "percentage", "ratio"}
+ and x_label
+ and len(metrics) == 1
+ and (series_labels or len(offsets) > 1)
+ and form_data.get("truncate_metric") is not None
+ and form_data.get("truncate_metric")
+ ):
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {
+ "columns": {metric_labels[0]: None},
+ "level": 0,
+ "inplace": True,
+ },
+ }
+ )
+ if complete_timeseries_contract:
+ x_axis_sort = form_data.get("x_axis_sort")
+ x_axis_sort_asc = form_data.get("x_axis_sort_asc")
+ sortable_labels = [
+ label
+ for label in [
+ x_label,
+ *(
+ _metric_label(metric)
+ for metric in _as_list(form_data.get("metrics"))
+ ),
+ *(
+ _metric_label(metric)
+ for metric in _timeseries_extra_metrics(form_data)
+ ),
+ ]
+ if label
+ ]
+ if (
+ x_axis_sort is not None
+ and x_axis_sort_asc is not None
+ and x_axis_sort in sortable_labels
+ and not _as_list(form_data.get("groupby"))
+ ):
+ options: dict[str, Any] = {"ascending": x_axis_sort_asc}
+ if x_axis_sort == x_label:
+ options["is_sort_index"] = True
+ else:
+ options["by"] = x_axis_sort
+ chain.append({"operation": "sort", "options": options})
+ chain.append({"operation": "flatten"})
+
+ if complete_timeseries_contract and form_data.get("forecastEnabled") and x_label:
+ x_axis_grain = (
+ x_axis.get("timeGrain")
+ if isinstance(x_axis, dict)
+ and x_axis.get("expressionType") in {"SIMPLE", "SQL"}
+ else None
+ )
+ time_grain = (
+ x_axis_grain
+ or (query.get("extras") or {}).get("time_grain_sqla")
+ or form_data.get("time_grain_sqla")
+ or "P1D"
+ )
+ chain.append(
+ {
+ "operation": "prophet",
Review Comment:
The Prophet step is now part of the compiled query for forecast charts, but the XY compile check still limits the query to 2 raw rows. For a saved forecast chart grouped by region where the two top rows are East and West on the same day, the pivot yields one timestamp and the forecast step fails, so a valid rebind or config update returns CHART_COMPILE_FAILED even though the full chart forecasts fine. Should the compile row limit for forecast-enabled charts be raised (or the forecast step skipped during compile) so it sees at least two timestamps?
##########
tests/unit_tests/mcp_service/chart/tool/test_update_chart_preview.py:
##########
@@ -1410,24 +1627,19 @@ async def test_validation_failure_skips_cache_write(
mock_create_form_data.assert_not_called()
@patch.object(update_chart_preview_module, "_find_dataset")
- @patch.object(update_chart_preview_module, "has_dataset_access", return_value=False)
- @patch("superset.daos.dataset.DatasetDAO.find_by_id")
@patch(
"superset.mcp_service.commands.create_form_data.MCPCreateFormDataCommand.run"
)
@pytest.mark.asyncio
async def test_dataset_access_denied_short_circuits(
self,
mock_create_form_data,
- mock_find_by_id,
- unused_access_mock,
mock_find_dataset,
mcp_server,
mock_auth,
):
- """has_dataset_access=False → DatasetNotAccessible, no cache write."""
- mock_find_dataset.return_value = _mock_dataset(id=3)
- mock_find_by_id.return_value = _mock_dataset(id=3)
+ """An inaccessible dataset short-circuits before mapping or cache writes."""
+ mock_find_dataset.return_value = None
Review Comment:
This test used to patch has_dataset_access to False and assert DatasetNotAccessible; it now mocks _find_dataset to return None, which never exercises the access check that now lives only inside _find_dataset (the tool's second check was removed). If _find_dataset stopped calling has_dataset_access, this test would still pass and an ungranted dataset could reach compilation. Could this keep _find_dataset real, mock DatasetDAO.find_by_id to return a dataset with has_dataset_access False, and assert success False with no compile or cache write?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,151 @@ def fallback_vega_lite_preview(
return None
+# Extended ISO date/datetime text that the renderer's ``Date.parse`` accepts.
+# Python's ISO parser also accepts basic forms such as ``20250101`` and week
+# dates, which browsers reject, so those must not mark the axis temporal.
+_VEGA_TEMPORAL_TEXT = re.compile(
+ r"\d{4}-\d{2}-\d{2}"
+ r"(?:[T ]\d{2}:\d{2}(?::\d{2}(?:\.\d{1,9})?)?(?:Z|[+-]\d{2}:\d{2})?)?"
+)
+
+
+def _xy_pivot_x_type(values: list[Any]) -> str:
+ """Infer the Vega-Lite x type from every x value, not a character scan.
+
+ The renderer has no column metadata, so text is temporal only when each
+ value is an extended ISO date or datetime the renderer can parse; labels
+ such as ``New York`` and compact dates such as ``20250101`` stay nominal
+ instead of becoming unparseable dates.
+ """
+ present = [value for value in values if value is not None]
+ if not present:
+ return "nominal"
+ if all(
+ isinstance(value, (int, float)) and not isinstance(value, bool)
+ for value in present
+ ):
+ return "quantitative"
+ if all(
+ isinstance(value, (date, datetime))
+ or (
+ isinstance(value, str)
+ and _VEGA_TEMPORAL_TEXT.fullmatch(value) is not None
+ and _gantt_temporal_value(value) is not None
+ )
+ for value in present
+ ):
+ return "temporal"
+ return "nominal"
+
+
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list, _time_comparison
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ def vega_field(name: str) -> str:
+ """Escape a literal key so Vega-Lite does not read it as a nested path."""
+ return "".join("\\" + char if char in ".[]\\" else char for char in name)
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ series_labels = list(metric_labels)
+ if len(metric_labels) == 1 and _time_comparison(
+ form_data, _as_list(form_data.get("metrics"))
+ ):
+ # A single compared metric renames each shifted series to its bare
+ # offset, so those series carry the offset rather than the metric.
+ series_labels.extend(
+ str(offset) for offset in _as_list(form_data.get("time_compare"))
+ )
+ prefixes = {
+ spelling
+ for label in series_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(prefix + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(prefix + "__")
+ for prefix in prefixes
+ )
+ ]
+ if not fields and len(metric_labels) == 1 and form_data.get("truncate_metric"):
+ # A single truncated metric drops its label from the pivoted column
+ # names, so every non-x-axis column is one category series.
+ fields = [field for field in data[0] if field != x_axis]
+ if not fields:
+ return None
+ x_type = _xy_pivot_x_type([row.get(x_axis) for row in data])
+ return VegaLitePreview(
+ specification={
+ "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
+ "data": {"values": data},
+ "transform": [
+ {
+ "fold": [vega_field(field) for field in fields],
+ "as": ["__mcp_xy_series", "__mcp_xy_value"],
+ }
+ ],
+ "mark": mark,
+ "encoding": {
+ "x": {"field": vega_field(x_axis), "type": x_type, "title": x_axis},
+ "y": {
Review Comment:
This folded-series spec encodes a quantitative y with a nominal color and never sets stack, so Vega-Lite stacks bars/areas by default. For an unstacked grouped bar chart (stack null, groupby region) with East=10 and West=20 at one x value, Explore shows separate bars at 10 and 20 but this preview shows one stacked bar at 30. Should the y encoding map the chart's saved stack control (and use stack: null for the unstacked case)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5524104520
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5561052111
Finite maintainer pass complete at 1e3150c8ae8d8cb5ed1ace38de6159cf7ac2558c. Both sadpandajoe threads were individually replied to and resolved after the normal push; no unresolved threads remain. All 13 required checks succeeded on this exact head (77 successful / 3 skipped / 2 neutral overall, no failed or pending checks). Local evidence: 4,414 full MCP tests, 450 focused tests, 427 shared backend tests, 122 frontend tests across 9 suites, and all applicable branch-wide pre-commit hooks passed. PR body contains commands, CI links, and the unrelated local declaration-build caveat. GitHub is MERGEABLE; BLOCKED reflects REVIEW_REQUIRED. No rebase/force-push, other-branch changes, reviewer spawn, or further independent review loop. Worktree clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3906299302
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1065,6 +1066,155 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ return ",".join(
+ format(value, "g") if isinstance(value, float) else value for value in values
+ )
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ metric_output = metric if isinstance(metric, str) else metric.get("label")
+ metric_targets = {
+ candidate.casefold()
+ for candidate in (
+ config.metric.name,
+ config.metric.label,
+ metric_output,
+ )
+ if candidate
+ }
+ dimensions = config.dimensions or []
+ dimension_targets = {
+ candidate.casefold(): dimension.name
+ for dimension in dimensions
+ for candidate in (dimension.name, dimension.label)
+ if candidate and dimension.name
+ }
+ form_data["orderby"] = [
+ [
+ (
+ metric
+ if order.column.casefold() in metric_targets
+ else dimension_targets[order.column.casefold()]
+ ),
+ order.ascending,
+ ]
+ for order in config.order_by
+ ]
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
Review Comment:
**Suggestion:** When only `time_range` changes and `temporal_column` is omitted, this preserves the old temporal adhoc filter, so the query continues using the previous time range. [logic error]
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Sometimes`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=e45ee7c0599d4b999c77b048f5a1e2f2&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=e45ee7c0599d4b999c77b048f5a1e2f2&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/mcp_service/chart/chart_utils.py
**Line:** 1214:1215
**Comment:**
*Logic Error: When only `time_range` changes and `temporal_column` is omitted, this preserves the old temporal adhoc filter, so the query continues using the previous time range.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=e7b59bb2eec1dd81f5256a9fa6f7c91bc1dbe04109613754c24dcaeb41f134fb&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=e7b59bb2eec1dd81f5256a9fa6f7c91bc1dbe04109613754c24dcaeb41f134fb&reaction=dislike'>👎</a>
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,272 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import BulletChartConfig, ColumnRef
+from superset.mcp_service.chart.validation.dataset_validator import (
+ DatasetValidator,
+ is_numeric_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ f"Ambiguous Bullet {role} {name!r}; exact-case matches are: "
+ f"{', '.join(matches)}"
+ )
+ return matches[0] if matches else name
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate(
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=f"Bullet {role} {name!r} is ambiguous by case",
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ column = next(
+ (
+ column
+ for column in dataset_context.available_columns
+ if column["name"].casefold() == (metric.name or "").casefold()
+ ),
+ None,
+ )
+ if column is None or is_numeric_column(column):
+ return None
Review Comment:
**Suggestion:** The numeric check matches only by casefolded name, so an exact-case metric can be validated against the wrong case-variant column when the dataset contains ambiguous names. [incorrect variable usage]
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Rarely`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=5b29a08035b14abdacef94ad70250107&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=5b29a08035b14abdacef94ad70250107&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/mcp_service/chart/plugins/bullet.py
**Line:** 169:177
**Comment:**
*Incorrect Variable Usage: The numeric check matches only by casefolded name, so an exact-case metric can be validated against the wrong case-variant column when the dataset contains ambiguous names.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=55f54e7f735580b061cce00f7b9e8183cf7ec82c764158526d62ce48bac7c7e3&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=55f54e7f735580b061cce00f7b9e8183cf7ec82c764158526d62ce48bac7c7e3&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4189857379
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1983 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
return None
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget, *, row_shaped: bool = False
+) -> str | None:
+ """Bound metadata, charging row-shaped indexes to the row-data work budget."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if row_shaped:
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "metadata contains too many total values"
+ else:
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list or type(item) is tuple:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = len(item)
+ # Chart Data emits one indexname per row. Only that outer array
+ # receives the row limit; containers within an index stay bounded.
+ max_items = (
+ MAX_QUERY_RESULT_ROWS
+ if row_shaped and depth == 0
+ else _MAX_ROW_CONTAINER_ITEMS
+ )
+ if width > max_items:
+ return "metadata contains an oversized array"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(width, mapping=False),
+ metadata=True,
+ ):
+ return reason
+ stack.extend((item[index], depth + 1, False) for index in range(width))
+ continue
+
+ normalized, reason = _normalize_trusted_scalar(
+ item, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ )
+ if reason is not None:
+ return f"metadata {reason}"
+ if reason := _charge_json_bytes(
+ budget,
+ _normalized_scalar_json_size(
+ normalized, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ ),
+ metadata=True,
+ ):
+ return reason
+ return None
+
+
+def _metadata_failure( # noqa: C901
+ payload: dict[str, Any], label: str
+) -> ChartError | None:
+ """Validate bounded cache and row-count metadata before any consumer."""
+ for key in ("rowcount", "total_rows"):
+ if key in payload and dict.__getitem__(payload, key) is not None:
+ try:
+ bounded_result_row_count(dict.__getitem__(payload, key))
+ except ValueError as ex:
+ return _malformed_result(f"{label} {key} {ex}")
+
+ if "is_cached" in payload:
+ is_cached = dict.__getitem__(payload, "is_cached")
+ if is_cached is not None and type(is_cached) is not bool:
+ return _malformed_result(f"{label} is_cached must be an exact boolean")
+
+ if "cache_key" in payload:
+ cache_key = dict.__getitem__(payload, "cache_key")
+ if cache_key is not None and (
+ type(cache_key) is not str
+ or _bounded_utf8_length(cache_key, _MAX_CACHE_STRING_BYTES) is None
+ ):
+ return _malformed_result(
+ f"{label} cache_key must be a bounded exact string"
+ )
+
+ # ChartData's production schema emits ``cached_dttm``. ``cache_dttm`` was
+ # used by earlier MCP payloads and remains a bounded compatibility alias.
+ # Validate both before cache utilities parse or compare either value.
+ for key in ("cached_dttm", "cache_dttm"):
+ if key not in payload:
+ continue
+ cache_dttm = dict.__getitem__(payload, key)
+ if cache_dttm is None:
+ continue
+ if type(cache_dttm) is str:
+ if _bounded_utf8_length(cache_dttm, _MAX_CACHE_STRING_BYTES) is not None:
+ continue
+ return _malformed_result(f"{label} {key} must be a bounded exact string")
+ if type(cache_dttm) is datetime:
+ tzinfo = cache_dttm.tzinfo
+ if tzinfo is None or any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ continue
+ return _malformed_result(f"{label} {key} has an unsupported timezone")
+ return _malformed_result(
+ f"{label} {key} must be a bounded exact string or datetime"
+ )
+ return None
+
+
+def _excel_temporal_cell(value: Any) -> Any | None:
+ """Return a naive builtin temporal value an XLSX writer stores natively.
+
+ Called only after ``_normalize_trusted_scalar`` accepted ``value``. XLSX has
+ no timezone-aware date cells, so aware values keep their ISO text, as do
+ NumPy durations whose unit may be ambiguous.
+ """
+ value_type = type(value)
+ if value_type is date or value_type is timedelta:
Review Comment:
P2: Excel's date system cannot represent values before 1900, so this now silently corrupts cells that previously exported as exact ISO text. In a local check through `_export_data_as_excel`, `datetime(1899, 12, 31, 5)` comes back as a bare `05:00` time in both openpyxl and xlsxwriter (the date is lost), xlsxwriter writes `date(1800, 1, 1)` as `1799-12-31`, and `timedelta(days=-2)` becomes `1899-12-28 00:00`. Could values outside Excel's representable range (pre-1900 dates/datetimes, negative durations) keep their ISO text, with a regression covering both writers?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4189474330
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1929 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
Review Comment:
Agreed: master wrote the raw Timestamp/date objects to the workbook. Fixed in edbce563ed (https://github.com/apache/superset/pull/43770/commits/edbce563ed): query_result_data takes preserve_excel_temporals (set only for format=excel); validation and size budgets still run on the JSON text, but naive row-level date/time/datetime/pandas Timestamp/datetime64/Timedelta cells are stored as builtin temporal values. Timezone-aware values keep ISO text (XLSX has no aware cells), and plugins with Chart Data temporal wire semantics (Bullet) keep their projection. The xlsxwriter fallback gets a default_date_format so those cells are dates there too. Regression test: test_excel_export_keeps_temporal_cells_as_dates[openpyxl|xlsxwriter].
##########
superset/mcp_service/chart/compile.py:
##########
@@ -284,6 +331,459 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ # Match QueryObject's guarded legacy saved-metric normalization.
+ if not ({"sqlExpression", "aggregate", "column"} & value.keys()):
+ label = value.get("label")
+ if isinstance(label, str) and label:
+ return "saved_metric", label
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
+ viz_type = form_data.get("viz_type")
+ if strict_all_form_refs and (
+ viz_type == "mixed_timeseries"
+ or (
+ isinstance(viz_type, str)
+ and (
+ viz_type.startswith("echarts_timeseries") or viz_type == "echarts_area"
+ )
+ )
+ ):
+ if (raw_x_axis := form_data.get("x_axis")) is not None and (
+ error := column_error(raw_x_axis, "form-data x_axis column")
+ ):
+ return error
+ metric_fields = [
+ "metrics",
+ "size",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ ]
+ if viz_type == "mixed_timeseries":
+ metric_fields.extend(
+ [
+ "metrics_b",
+ "size_b",
+ "timeseries_limit_metric_b",
+ "series_limit_metric_b",
+ ]
+ )
+ for field_name in metric_fields:
+ raw_value = form_data.get(field_name)
+ values = raw_value if isinstance(raw_value, list) else [raw_value]
+ for value in values:
+ if value is not None and (
+ error := metric_error(value, f"form-data {field_name} metric")
+ ):
+ return error
+
+ layer_suffixes = ("", "_b") if viz_type == "mixed_timeseries" else ("",)
+ for suffix in layer_suffixes:
+ sort_field = f"x_axis_sort{suffix}"
+ if sort_field not in form_data or form_data.get(sort_field) is None:
+ continue
+ x_axis = form_data.get(f"x_axis{suffix}", form_data.get("x_axis"))
+ allowed_labels: set[str] = set()
+ if x_axis_label := _native_column_label(x_axis):
+ allowed_labels.add(x_axis_label)
+ raw_metrics = form_data.get(f"metrics{suffix}")
+ for metric in raw_metrics if isinstance(raw_metrics, list) else []:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ raw_limit_metric = form_data.get(f"timeseries_limit_metric{suffix}")
+ limit_metrics = (
+ raw_limit_metric
+ if isinstance(raw_limit_metric, list)
+ else [raw_limit_metric]
+ )
+ for metric in limit_metrics:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ sort_value = form_data[sort_field]
+ if not isinstance(sort_value, str) or sort_value not in allowed_labels:
+ return _native_validation_error(sort_field, repr(sort_value)[:200])
+
+ if (
+ strict_all_form_refs
+ and isinstance(viz_type, str)
+ and viz_type.startswith("deck_")
+ ):
+ # Deck layers store most query roles outside common columns/metrics.
+ # Validate every renderer-consumed raw control as well as the generated
+ # QueryObject so a dataset-only rebind cannot hide or discard a stale
+ # tooltip, cross-filter, spatial, path, or metric reference.
+ for spatial_field in ("spatial", "start_spatial", "end_spatial"):
+ spatial = form_data.get(spatial_field)
+ if spatial is None:
+ continue
+ if not isinstance(spatial, dict):
+ return _native_validation_error(
+ f"form-data {spatial_field}", repr(spatial)[:200]
+ )
+ spatial_type = spatial.get("type")
+ if not isinstance(spatial_type, str):
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ role_fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if role_fields is None:
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ for role_field in role_fields:
+ spatial_value = spatial.get(role_field)
+ if spatial_value is None:
+ return _native_validation_error(
+ f"form-data {spatial_field}.{role_field} column", "missing"
+ )
+ if error := column_error(
+ spatial_value, f"form-data {spatial_field}.{role_field} column"
+ ):
+ return error
+
+ for field_name in (
+ "line_column",
+ "geojson",
+ "dimension",
+ "cross_filter_column",
+ ):
+ column_value = form_data.get(field_name)
+ if column_value is not None and (
+ error := column_error(column_value, f"form-data {field_name} column")
+ ):
+ return error
+
+ tooltip_contents = form_data.get("tooltip_contents")
+ if tooltip_contents is not None and not isinstance(tooltip_contents, list):
+ return _native_validation_error(
+ "form-data tooltip_contents", repr(tooltip_contents)[:200]
+ )
+ for index, item in enumerate(tooltip_contents or []):
+ tooltip_value: Any = None
+ if isinstance(item, str):
+ tooltip_value = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ tooltip_value = item.get("column_name")
+ if tooltip_value is not None and (
+ error := column_error(
+ tooltip_value, f"form-data tooltip_contents[{index}] column"
+ )
+ ):
+ return error
+
+ metric_values: list[tuple[str, Any]] = []
+ if viz_type not in {"deck_geojson", "deck_polygon"}:
+ for field_name in ("metrics", "metric", "size"):
+ raw_deck_metrics = form_data.get(field_name)
+ deck_metrics = (
+ raw_deck_metrics
+ if isinstance(raw_deck_metrics, list)
+ else [raw_deck_metrics]
+ )
+ metric_values.extend(
+ (f"form-data {field_name} metric", deck_metric)
+ for deck_metric in deck_metrics
+ if deck_metric is not None
+ )
+ if viz_type == "deck_polygon" and form_data.get("metric") is not None:
+ metric_values.append(("form-data metric metric", form_data.get("metric")))
+ fixed_metric_fields = (
+ ("point_radius_fixed",)
+ if viz_type in {"deck_scatter", "deck_polygon"}
+ else ()
+ ) + (("line_width",) if viz_type == "deck_path" else ())
+ for field_name in fixed_metric_fields:
+ fixed_value = form_data.get(field_name)
+ deck_metric: Any = (
+ fixed_value
+ if (
+ isinstance(fixed_value, str)
+ and fixed_value
+ and viz_type != "deck_polygon"
+ )
+ else None
+ )
+ if isinstance(fixed_value, dict) and fixed_value.get("type") == "metric":
+ deck_metric = fixed_value.get("value")
+ if deck_metric is not None:
+ metric_values.append((f"form-data {field_name} metric", deck_metric))
+ if viz_type == "deck_path" and form_data.get("breakpoint_metric") is not None:
+ metric_values.append(
+ (
+ "form-data breakpoint_metric metric",
+ form_data.get("breakpoint_metric"),
+ )
+ )
+ for role, deck_metric in metric_values:
+ if error := metric_error(deck_metric, role):
+ return error
+
+ for filter_ in form_data.get("adhoc_filters") or []:
+ if not isinstance(filter_, dict) or filter_.get("expressionType") != "SIMPLE":
+ continue
+ if not strict_all_form_refs and _is_inert_adhoc_filter(filter_):
+ continue
+ subject = filter_.get("subject")
+ clause = str(filter_.get("clause") or "WHERE").upper()
+ if clause == "HAVING" and isinstance(subject, str):
+ metric_matches = (
+ [subject]
+ if subject in saved_metrics
+ else [
+ name
+ for name in saved_metrics
+ if name.casefold() == subject.casefold()
+ ]
+ )
+ if len(metric_matches) == 1:
+ continue
+ if subject is not None and (
+ error := column_error(subject, "form-data filter column")
+ ):
+ return error
+ if filter_.get("operator") == "TEMPORAL_RANGE" and isinstance(subject, str):
+ try:
+ temporal = resolve_dataset_column(subject, dataset_context)
+ except ValueError:
+ temporal = None
+ if temporal is not None and not temporal.get("is_temporal", False):
+ return _native_validation_error("temporal filter column", subject)
+
+ # temporal_columns_lookup describes the entire datasource, not selected
+ # roles. The physical form/query column checks validate selected references.
+
+ for query_index, query in enumerate(queries, 1):
+ metric_labels: set[str] = set()
+ for column in query.get("columns") or []:
+ if error := column_error(column, f"query {query_index} column"):
+ return error
+ for column in query.get("series_columns") or []:
+ if error := column_error(column, f"query {query_index} series column"):
+ return error
+ for column in query.get("groupby") or []:
+ if error := column_error(column, f"query {query_index} groupby column"):
+ return error
+ selected_column_labels = {
+ label
+ for column in query.get("columns") or []
+ if (label := _native_column_label(column)) is not None
+ }
+ for level in query.get("grouping_sets") or []:
+ for column in level:
+ # Grouping sets reference selected logical outputs, including
+ # Custom SQL labels. Their source columns were checked above.
+ if isinstance(column, str) and column in selected_column_labels:
+ continue
+ if error := column_error(
+ column, f"query {query_index} grouping-set column"
+ ):
+ return error
+
+ metrics = query.get("metrics") or []
+ for metric in metrics:
+ if label := _metric_label_for_validation(metric):
+ metric_labels.add(label)
+ if error := metric_error(metric, f"query {query_index} metric"):
+ return error
+
+ granularity = query.get("granularity")
+ if granularity:
+ if error := column_error(granularity, "temporal column"):
+ return error
+ try:
+ temporal = resolve_dataset_column(granularity, dataset_context)
+ except ValueError:
+ temporal = None
+ if (
+ (query.get("extras") or {}).get("time_grain_sqla")
+ and temporal is not None
+ and not temporal.get("is_temporal", False)
+ ):
+ return _native_validation_error("temporal column", granularity)
+
+ for filter_ in query.get("filters") or []:
+ if not isinstance(filter_, dict):
+ return _native_validation_error("filter", repr(filter_)[:200])
+ column = filter_.get("col")
+ if isinstance(column, str) and column in metric_labels:
+ continue
+ if isinstance(column, str) and any(
+ name.casefold() == column.casefold() for name in saved_metrics
+ ):
+ continue
+ if column is not None and (
+ error := column_error(column, f"query {query_index} filter column")
+ ):
+ return error
+
+ for order in query.get("orderby") or []:
+ if not isinstance(order, (list, tuple)) or len(order) != 2:
+ return _native_validation_error("ordering", repr(order)[:200])
+ target = order[0]
+ target_label = _metric_label_for_validation(target)
+ if target in metrics or (target_label and target_label in metric_labels):
+ continue
+ if isinstance(target, str) and target in metric_labels:
+ continue
+ # Native ordering can use a metric that is not displayed. Resolve
+ # saved names with the same exact/case-folded rules as other metrics,
+ # and validate adhoc metric columns rather than their output labels.
+ if (
+ isinstance(target, str)
+ and any(name.casefold() == target.casefold() for name in saved_metrics)
+ ) or (isinstance(target, dict) and _native_metric_ref(target) is not None):
+ if error := metric_error(
+ target, f"query {query_index} ordering metric"
+ ):
+ return error
Review Comment:
Agreed: get_sqla_query checks adhoc_columns_by_label before columns_by_name. Fixed in edbce563ed (https://github.com/apache/superset/pull/43770/commits/edbce563ed): native reference validation accepts a string sort that matches a labelled adhoc column of the same query; an unknown alias still fails closed. Regression tests: test_saved_bullet_labels_update_keeps_sql_dimension_label_sort and test_bullet_native_validation_resolves_sort_by_adhoc_column_label.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,610 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if type(value) in {list, tuple, dict}:
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(
+ {"queries": [{"data": [{"value": value}]}]}, temporal_json_numbers=True
+ )
+ if failure is not None or data is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} "
+ "has an invalid or unbounded container value"
+ )
+ normalized = data[0][0]["value"]
+ reason = None
+ elif _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if value_type in {list, dict}:
+ text = _bullet_container_category_text(normalized, dimension, row_index)
+ elif normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_container_category_text(value: Any, dimension: str, row_index: int) -> str:
+ """Stringify validated containers like JavaScript with a bounded text budget."""
+ from superset.mcp_service.chart.query_result import _bounded_utf8_length
+
+ if type(value) is dict:
+ return "[object Object]"
+ parts: list[str] = []
+ size = 0
+ for index, item in enumerate(value):
+ if type(item) in {list, dict}:
+ text = _bullet_container_category_text(item, dimension, row_index)
+ else:
+ text = (
+ ""
+ if item is None
+ else _bullet_category_value(item, dimension, row_index)[1]
+ )
+ text_size = _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES)
+ size += (text_size if text_size is not None else _MAX_BULLET_TEXT_BYTES + 1) + (
+ index > 0
+ )
+ if size > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ parts.append(text)
+ return ",".join(parts)
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ from superset.utils.number_format import D3_FORMAT_RE
+
+ # Specifier length does not bound its requested output precision. Check the
+ # parsed precision before the formatter can allocate or round any value.
+ match = D3_FORMAT_RE.match(format_)
+ if match and match.group(8) is not None and int(match.group(8)) > 20:
+ raise BulletOutputError(
+ "Bullet number format precision must not exceed 20",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet number format {format_!r} is unsupported by previews",
+ error_type="UnsupportedFormat",
+ ) from ex
+ return format_
+
+
+def _format_bullet_number(format_: str, value: float) -> str:
+ """Format finite Bullet values, including the full binary-float range."""
+ from superset.utils.number_format import format_numeric
+
+ try:
+ # Binary64 has at most 309 integer digits. Leave room for the bounded
+ # fractional precision, percent scaling, and a rounding carry without
+ # changing the caller's Decimal context.
+ with localcontext() as context:
+ context.prec = max(context.prec, 334)
+ return format_numeric(format_, value)
+ except OverflowError:
+ # SMART_NUMBER's significant-digit rounding can overflow a finite float
+ # near DBL_MAX. Scientific repr remains deterministic and informative.
+ if format_ in {"SMART_NUMBER", "SMART_NUMBER_SIGNED"} and math.isfinite(value):
+ prefix = "+" if format_ == "SMART_NUMBER_SIGNED" and value > 0 else ""
+ return prefix + repr(value)
+ raise
+
+
+def _containing_bullet_range_label(
+ measure: float, ranges: list[float], labels: list[str]
+) -> str | None:
+ """Match the frontend's labelled containing-range tooltip selection."""
+ ascending = sorted(
+ (
+ (value, labels[index] if index < len(labels) else "")
+ for index, value in enumerate(ranges)
+ ),
+ key=lambda entry: entry[0],
+ )
+ for threshold, label in ascending:
+ if measure <= threshold:
+ return label or None
+ if ascending and ascending[-1][1]:
+ return f"> {ascending[-1][1]}"
+ return None
+
+
+def resolve_bullet_render_model( # noqa: C901
+ data: List[Dict[str, Any]],
+ form_data: Dict[str, Any],
+ *,
+ validate_format: bool = True,
+) -> BulletRenderModel:
+ """Resolve Bullet rows; check preview formatter support only when rendering."""
+ if type(data) is not list:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ for row_index in range(list.__len__(data)):
+ row = list.__getitem__(data, row_index)
+ if type(row) is not dict:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ if dict.__len__(row) > _MAX_BULLET_FIELDS:
+ raise BulletOutputError("Bullet query row exceeds the field limit")
+ for key in dict.keys(row):
+ if type(key) is not str:
+ raise BulletOutputError("Bullet query row keys must be strings")
+ if len(key) > _MAX_BULLET_FIELD_BYTES:
+ raise BulletOutputError("Bullet query row key exceeds the size limit")
+
+ if type(form_data) is not dict:
+ raise BulletOutputError("Bullet form data must be an object")
+
+ metric_label = _form_metric_label(dict.get(form_data, "metric"))
+ if not metric_label:
+ raise BulletOutputError("Bullet metric has no declared result alias")
+ raw_groupby = dict.get(form_data, "groupby")
+ if raw_groupby is None:
+ raw_groupby = []
+ if type(raw_groupby) is not list:
+ raise BulletOutputError("Bullet dimensions must be an array")
+ dimension_labels = [
+ _form_column_label(list.__getitem__(raw_groupby, index))
+ for index in range(list.__len__(raw_groupby))
+ ]
+ if any(not label for label in dimension_labels):
+ raise BulletOutputError("Bullet dimension has no declared result alias")
+
+ if data:
+ first_row = list.__getitem__(data, 0)
+ metric_field = _require_result_field(metric_label, first_row, "metric")
+ dimensions = [
+ _require_result_field(label, first_row, "dimension")
+ for label in dimension_labels
+ ]
+ else:
+ # The frontend accepts empty results. Ungrouped charts retain one
+ # zero-valued measure; grouped charts retain the declared roles but no
+ # categories or rows are fabricated.
+ metric_field = metric_label
+ dimensions = [label for label in dimension_labels if label is not None]
+
+ measures: list[float] = []
+ copied_rows: list[dict[str, Any]] = []
+ for index in range(list.__len__(data)):
+ row = list.__getitem__(data, index)
+ row_metric_field = _require_result_field(
+ metric_label, row, f"metric row {index}"
+ )
+ measure = _bullet_number(
+ dict.__getitem__(row, row_metric_field), index, metric_field
+ )
+ # Reserve every exact output key so the internal Vega category alias
+ # cannot collide with an unselected result field. Unselected values are
+ # deliberately replaced with None rather than converted or serialized.
+ copied: dict[str, Any] = dict.fromkeys(dict.keys(row))
+ copied[metric_field] = measure
+ for label, dimension in zip(dimension_labels, dimensions, strict=True):
+ row_dimension = _require_result_field(label, row, f"dimension row {index}")
+ dimension_value, _ = _bullet_category_value(
+ dict.__getitem__(row, row_dimension), dimension, index
+ )
+ copied[dimension] = dimension_value
+ copied_rows.append(copied)
+ measures.append(measure)
+
+ # The frontend validates/coerces the whole result array but renders only
+ # the first row for an ungrouped aggregate.
+ if not dimensions:
+ copied_rows = copied_rows[:1]
+ measures = measures[:1]
+ if not copied_rows:
+ copied_rows = [{metric_field: 0.0}]
+ measures = [0.0]
+
+ ranges = _bullet_numeric_control_tokens(dict.get(form_data, "ranges"), "ranges")
+ if not ranges:
+ # Match Bullet/transformProps.ts: the largest measure drives one
+ # qualitative band whose upper threshold is 110% of that measure.
+ ranges = [0.0, max(measures, default=0.0) * 1.1]
+ markers = _bullet_numeric_control_tokens(dict.get(form_data, "markers"), "markers")
+ marker_lines = _bullet_numeric_control_tokens(
+ dict.get(form_data, "marker_lines"), "marker lines"
+ )
+ all_numbers = [*measures, *ranges, *markers, *marker_lines]
+ if any(not math.isfinite(value) for value in all_numbers):
+ raise BulletOutputError("Bullet presentation values must be finite")
+
+ show_labels = dict.get(form_data, "show_labels", False)
+ show_legend = dict.get(form_data, "show_legend", False)
+ if type(show_labels) is not bool or type(show_legend) is not bool:
+ raise BulletOutputError("Bullet label and legend controls must be booleans")
+
+ range_labels = _bullet_string_tokens(dict.get(form_data, "range_labels"))
+ marker_labels = _bullet_string_tokens(dict.get(form_data, "marker_labels"))
+ marker_line_labels = _bullet_string_tokens(
+ dict.get(form_data, "marker_line_labels")
+ )
+ return BulletRenderModel(
+ rows=copied_rows,
+ metric_field=metric_field,
+ dimensions=dimensions,
+ measures=measures,
+ ranges=ranges,
+ range_labels=range_labels,
+ markers=markers,
+ marker_labels=marker_labels,
+ marker_lines=marker_lines,
+ marker_line_labels=marker_line_labels,
+ y_axis_format=(
+ _validate_bullet_format(
+ dict.get(form_data, "y_axis_format", "SMART_NUMBER"), all_numbers
+ )
+ if validate_format
+ else "SMART_NUMBER"
+ ),
+ show_labels=show_labels,
+ show_legend=show_legend,
+ )
+
+
+def _generate_ascii_bullet_chart(
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> str:
+ """Generate a horizontal Bullet preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+ if model.dimensions and not model.rows:
+ return "No data available for grouped Bullet chart"
+ extent = (
+ max(
+ [abs(value) for value in [*model.measures, *model.ranges, *model.markers]],
+ default=1,
+ )
+ or 1
+ )
+ lines = [f"ASCII Bullet Chart — {model.metric_field}", "=" * 60]
+ for row_index, (row, value) in enumerate(
+ zip(model.rows[:10], model.measures[:10], strict=True)
+ ):
+ category = ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ category = category or "Measure"
+ width = round(abs(value) / extent * 32)
+ bar = "█" * width
+ formatted = _format_bullet_number(model.y_axis_format, value)
+ lines.append(f"{category[:20]:>20} |{bar:<32} {formatted}")
+
+ def labeled(values: list[float], labels: list[str], prefix: str) -> list[str]:
+ return [
+ f"{labels[index] if index < len(labels) and labels[index] else prefix}: "
+ f"{_format_bullet_number(model.y_axis_format, value)}"
+ for index, value in enumerate(values)
+ ]
+
+ if model.show_labels or model.show_legend or model.marker_lines:
+ lines.append("Key:")
+ if model.show_labels or model.show_legend:
Review Comment:
Agreed: the frontend always draws range bands and markers; show_labels/show_legend only toggle their text. Fixed in edbce563ed (https://github.com/apache/superset/pull/43770/commits/edbce563ed): the ASCII preview always lists ranges, markers and marker lines under Key. Regression test: test_bullet_ascii_shows_targets_with_default_label_toggles (goal 60 vs 80 with default toggles).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4194170734
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -286,7 +2244,8 @@ def normalize_chart_query_result(result: Any, form_data: Mapping[str, Any]) -> A
def normalize_treemap_query_result(result: Any, form_data: Mapping[str, Any]) -> Any:
"""Require unique hierarchy outputs and finite numeric Treemap metrics."""
- if failure := query_result_failure(result):
+ _data, failure = query_result_data(result, preserve_nonfinite_floats=True)
Review Comment:
Fixed in 10d39d282631e2a307996fc25609b2b75f403368. Treemap validation inspects detached normalized rows without rewriting the source envelope, leaving CSV/Excel normalization format-aware. Regression tests cover saved, saved-cache, and unsaved-cache exports: CSV keeps the datetime space separator and Excel retains native date cells. Nested-container copy and validation-bound tests also pass.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2445,92 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding exact output keys avoids splitting category values that contain
+ escaped commas. The legend retains each complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ escape_separator(label)
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(label + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(label + "__")
+ for label in metric_labels
+ )
+ ]
+ if not fields:
+ return None
+ sample = data[0][x_axis]
+ x_type = (
+ "temporal"
+ if isinstance(sample, str) and any(char in sample for char in "-/: ")
+ else "quantitative"
+ if isinstance(sample, (int, float))
+ else "nominal"
+ )
+ return VegaLitePreview(
+ specification={
+ "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
+ "data": {"values": data},
+ "transform": [
+ {"fold": fields, "as": ["__mcp_xy_series", "__mcp_xy_value"]}
Review Comment:
Fixed in 10d39d282631e2a307996fc25609b2b75f403368. Fold field paths escape dots, brackets, and backslashes while the data keys stay literal. Regression tests exercise dotted, bracketed, and backslash category labels through saved and unsaved line/bar preview dispatch.
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1349,6 +1362,118 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Inherited native SQL dimensions remain query expressions when updating metrics
+or `order_by`; reference their output labels to sort by them. Caller-supplied
+`dimensions` must still be physical columns, not SQL expressions.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling. Bullet metric strings use
+JavaScript numeric spellings: underscore separators and non-ASCII digits are
+rejected rather than interpreted as numbers. Bounded array-valued dimensions
+retain their raw values in data reads and exports; preview category labels use
+JavaScript string conversion (for example, `[1, 2]` displays as `1,2`). Native
+SQL metrics without a label use their SQL expression as the output label.
+
+Presentation-only updates preserve saved predicates, including legacy top-level
+`where`, `having`, and `filters`, and normalize the native `order_by_cols` alias.
+When both `orderby` and `order_by_cols` are saved, their sort entries are
+concatenated in form-data key order, including when `orderby` is empty.
+Explicit `filters: []` and `order_by: []` clear those inherited controls.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+Labels that become empty after sanitization retain their slots so later labels
+remain aligned with their corresponding values.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates. Omitted label controls preserve the saved state only when their
+corresponding value controls (`ranges`, `markers`, or `marker_lines`) are also
+omitted. Replacing a value control without its labels clears the saved labels;
+resend the labels to retain annotations with the replacement values.
+
+Saved native `ranges`, `markers`, and `marker_lines` controls ignore empty,
+non-numeric, and NaN tokens, matching Explore. If no numeric range remains, the
+preview uses the default band up to 110% of the largest measure. Unrelated
+updates preserve these saved controls. Newly authored typed lists must contain
+finite numbers; infinite native values and oversized controls remain errors.
+
+On same-dataset chart updates, `filters: []` clears both user filters and the
+generated dashboard-time binding. `temporal_column: null` clears only that
+generated binding, preserving user filters. Omitting those controls preserves
+the saved binding. Bullet Vega previews keep separate indexed rows even when
+dimension display labels are identical, and support both `SMART_NUMBER` and
+`SMART_NUMBER_SIGNED` number formats. Bullet preview numeric format
+precision is limited to 20 digits before formatting; raw data reads do not
+validate presentation formats.
+
+Native Bullet temporal filters retain the active filter's subject and range as a
+pair; `No filter` placeholders do not supply the subject of another active range.
+Typed configs support one such pair. Multiple active native temporal filters, or
+conflicts with explicit `temporal_column`/`time_range`, are rejected rather than
+silently dropping or moving a predicate.
+
+### Query result limits
+
+MCP chart tools accept up to 50,000 rows per query. The row-shaped `indexnames`
Review Comment:
Fixed in 10d39d282631e2a307996fc25609b2b75f403368. The MCP documentation and UPDATING.md describe the 100,000-total-row and 2,500,000-value/container budgets, how nested values, repeated occurrences, object keys, and indexnames are counted, and why 50,000 rows with 50 scalar columns exceed the value budget.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1177,372 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
Review Comment:
Fixed in 10d39d282631e2a307996fc25609b2b75f403368. Removed _bullet_numeric_tokens and _canonical_result_field after a repository-wide Python search confirmed neither had callers. Active strict control parsing and result-alias resolution are unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4180929591
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,465 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ temporal_pairs: list[tuple[str, str]] = []
+ inert_subjects: set[str] = set()
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ if not isinstance(comparator, str) or not comparator:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs a range"
+ )
+ if comparator.casefold() == "no filter":
+ inert_subjects.add(subject)
+ else:
+ temporal_pairs.append((subject, comparator))
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ if len(temporal_pairs) > 1:
+ raise ValueError(
+ "Multiple active native temporal filters cannot be represented by "
+ "a single temporal_column/time_range pair"
+ )
+ if temporal_pairs:
+ subject, comparator = temporal_pairs[0]
+ if data.get("temporal_column") not in (None, subject) or data.get(
+ "time_range"
+ ) not in (None, "No filter", comparator):
+ raise ValueError(
+ "Native temporal filter conflicts with temporal_column/time_range"
+ )
+ data["temporal_column"] = subject
+ data["time_range"] = comparator
+ elif len(inert_subjects) == 1:
+ data.setdefault("temporal_column", next(iter(inert_subjects)))
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ from superset.mcp_service.chart.preview_utils import (
+ _bullet_numeric_control_tokens,
+ )
+
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ # Saved Explore controls are lenient; newly authored typed lists
+ # still undergo the strict List[float] validation below.
+ for control in ("ranges", "markers", "marker_lines"):
+ if control in data:
+ data[control] = _bullet_numeric_control_tokens(
+ data[control], control
+ )
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
+ {"name": item} if isinstance(item, str) else item
+ for item in data[key]
+ ]
+ for key in ("order_by", "orderby", "order_by_cols"):
+ if key in data:
+ data[key] = cls._adapt_native_order_by(data[key])
+ cls._adapt_native_filters(data)
+ return data
+
+ @field_validator("ranges", "markers", "marker_lines", mode="before")
+ @classmethod
+ def tokenize_native_numeric_lists(cls, value: Any) -> Any:
+ """Parse numeric controls without creating values for empty tokens."""
+ if value is None:
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",") if token.strip()]
+ return value
+
+ @field_validator(
+ "range_labels", "marker_labels", "marker_line_labels", mode="before"
+ )
+ @classmethod
+ def tokenize_native_label_lists(cls, value: Any) -> Any:
+ """Parse label controls while preserving positional empty tokens."""
+ if value is None or value == "":
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",")]
+ return value
+
+ @field_validator("ranges", "markers", "marker_lines")
+ @classmethod
+ def reject_non_finite_values(cls, values: List[float]) -> List[float]:
+ if any(not math.isfinite(value) for value in values):
+ raise ValueError("Bullet thresholds and markers must be finite numbers")
+ return values
+
+ @field_validator("range_labels", "marker_labels", "marker_line_labels")
+ @classmethod
+ def validate_presentation_labels(cls, labels: List[str]) -> List[str]:
+ result: list[str] = []
+ for label in labels:
+ if "," in label:
+ raise ValueError(
+ "Bullet labels cannot contain commas because the frontend "
+ "comma-separated controls have no escaping"
+ )
+ if label == "":
+ result.append("")
+ continue
+ sanitized = sanitize_user_input(
+ label, "Bullet label", max_length=200, allow_empty=True
+ )
+ if sanitized is not None:
Review Comment:
Fixed in 749076630cfd8f3f814c6860ac96f722d9174882: sanitized-empty range, marker, and marker-line labels retain empty strings so later labels stay aligned with their values.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5861139090
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5837264074
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5768491305
Fixed the dimension-less Bullet acceptance failure in fdfc3d6632635b51426b6b470a54081bd11bd9f8.
### Confirmed cause
Direct `BulletChartConfig.model_validate()` succeeds when `dimensions` is omitted. However, `generate_chart` then called `request.model_dump()` before the validation pipeline, inserting `dimensions: None` and all other defaults. Revalidation rejected that value with the reported `config -> bullet: ... dimensions must be an array` error.
No `groupby` was injected in this reproduction: it failed in the adapter's later array-conversion loop, **not** the two-alias conflict check. Merely changing the conflict guard would therefore have left the simplest request broken.
### Fix and scope
- Preserve omitted fields, including nested controls, with `exclude_unset=True` at the generation-to-validation boundary. Retain `chart_type` explicitly for typed callers relying on its default discriminator.
- Treat null `dimensions`/`groupby` as absent before Bullet alias checking and adaptation. Omitted/null hierarchies preserve saved state; `dimensions: []` still explicitly clears it. Conflicting non-null aliases remain errors.
- This serialization exposure is **shared by all 15 typed configs**, not just Bullet. Auditing all 22 published examples found Gauge, xy and the other types tolerate their dumped defaults, but the dump still loses omission provenance. The boundary fix applies to all types; added Gauge/xy/table/Bullet tests include nested axis/table controls.
- `update_chart` does not use the same full-dump boundary. Its omitted-dimensions case passed the local pre-fix tool test; explicit null failed before and passes after. Updated regressions cover omitted/null input against both empty and populated saved hierarchies.
### Verification
The new `test_generate_chart_creates_bullet_with_dimensions_omitted` omits the key entirely, exercises real schema validation and dataset normalization, and asserts successful creation and the payload sent to the mocked database command. **Confirmed fails before the fix with the reported validation error; passes afterward.** It does not substitute `dimensions: []`.
279 Bullet/generate-chart tests pass; staged-file pre-commit and explicit mypy pass. Full MCP testing and CI for the new head are being checked. Documentation explains omission/null versus explicit clearing. No frontend rendering behavior changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5864955561
Merged master without rewriting history; retained Bullet validation and preserved the upstream gauge and Unicode-ID regression coverage. Updated test doubles for the consolidated dataset lookup and query-context serialization. Tests: 239 Bullet tests and 86 generate/update-preview tests passed. Live MCP HTTP generation, update, compile, supported previews, data, SQL/export, and schema/examples passed; the table preview explicitly reports UnsupportedFormat. Changed-test pre-commit passed. Branch-wide Python checks and frontend lint passed after resolving setup; local frontend type checking remains blocked by TS6305 missing generated project-reference declarations. CI and exact-head independent review remain required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4175612738
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,454 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
Review Comment:
For a semantic-view Table grouped by a temporal column with a daily grain, this `BASE_AXIS` object lacks `isColumnReference: true`, so the semantic mapper rejects it as an unsupported adhoc dimension and MCP previews/data retrieval fail. Could this preserve the semantic-source flag emitted by the frontend Table builder?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1888 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+ return _ErrorText(text="; ".join(parts) or None)
+
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
return None
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
Review Comment:
A trusted `np.longdouble("1.000000000000000001")` metric now becomes an exact decimal string, but downstream Treemap and Gauge validation still reject strings as nonnumeric, so previously valid compile, preview, and Treemap data/export requests fail. Could those numeric consumers remain compatible with the new exact wire representation?
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,434 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
Review Comment:
A saved Table metric in the legacy `{"label": "Revenue"}` format is supported by `QueryObject._set_metrics`, but this rejects it before a dataset rebind even when the replacement dataset exposes `Revenue`. Could native reference validation apply the same guarded legacy normalization so compatible saved charts remain updatable?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1097,364 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = value.split(",")
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str:
+ token = token.strip(_JAVASCRIPT_WHITESPACE)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str:
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", token):
+ token = int(token, 0)
+ elif not re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ token,
+ ):
+ continue
+ try:
+ number = float(token)
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ metric_reference = "".join(
+ "\\" + char if char in ".[]\\" else char for char in model.metric_field
+ )
+ category_field = _unique_bullet_category_field(model.rows)
+ row_field = _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_row", (category_field,)
+ )
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field, row_field)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[row_field] = row_index
+ copied[category_field] = (
+ ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ if model.dimensions
+ else ""
+ )
+ if (
+ range_tooltip_field is not None
+ and containing_range_labels[row_index] is not None
+ ):
+ copied[range_tooltip_field] = containing_range_labels[row_index]
+ values.append(copied)
+
+ # Explore uses indexed rows even when their display labels are identical.
+ category_labels = [row[category_field] for row in values]
+ vega_format = {
+ "SMART_NUMBER": "~s",
+ "SMART_NUMBER_SIGNED": "+~s",
+ }.get(model.y_axis_format, model.y_axis_format)
+ y_encoding = {
+ "field": row_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ "sort": None,
+ "axis": {"labelExpr": f"{json.dumps(category_labels)}[datum.value]"},
+ }
+ tooltip = [
+ {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ },
+ {
+ "field": metric_reference,
+ "type": "quantitative",
+ "title": model.metric_field,
+ "format": vega_format,
+ },
+ ]
+ if range_tooltip_field is not None:
+ tooltip.append(
+ {"field": range_tooltip_field, "type": "nominal", "title": "Range"}
+ )
+ axis_min = min(
+ 0.0,
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ axis_max = max(
Review Comment:
With zero rows, `groupby: ["Region"]`, a single range such as `"100"`, and no markers, this becomes `max(100.0)` and raises `TypeError` instead of returning an empty Vega preview. Could this pass an iterable to `max` so the one-threshold empty-result case works too?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1888 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+ return _ErrorText(text="; ".join(parts) or None)
+
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
return None
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget, *, row_shaped: bool = False
+) -> str | None:
+ """Bound metadata, charging row-shaped indexes to the row-data work budget."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if row_shaped:
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "metadata contains too many total values"
+ else:
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = list.__len__(item)
+ # Chart Data emits one indexname per row. Only that outer array
+ # receives the row limit; containers within an index stay bounded.
+ max_items = (
+ MAX_QUERY_RESULT_ROWS
+ if row_shaped and depth == 0
+ else _MAX_ROW_CONTAINER_ITEMS
+ )
+ if width > max_items:
+ return "metadata contains an oversized array"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(width, mapping=False),
+ metadata=True,
+ ):
+ return reason
+ stack.extend(
+ (list.__getitem__(item, index), depth + 1, False)
+ for index in range(width)
+ )
+ continue
+
+ normalized, reason = _normalize_trusted_scalar(
+ item, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ )
+ if reason is not None:
+ return f"metadata {reason}"
+ if reason := _charge_json_bytes(
+ budget,
+ _normalized_scalar_json_size(
+ normalized, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ ),
+ metadata=True,
+ ):
+ return reason
+ return None
+
+
+def _metadata_failure( # noqa: C901
+ payload: dict[str, Any], label: str
+) -> ChartError | None:
+ """Validate bounded cache and row-count metadata before any consumer."""
+ for key in ("rowcount", "total_rows"):
+ if key in payload and dict.__getitem__(payload, key) is not None:
+ try:
+ bounded_result_row_count(dict.__getitem__(payload, key))
+ except ValueError as ex:
+ return _malformed_result(f"{label} {key} {ex}")
+
+ if "is_cached" in payload:
+ is_cached = dict.__getitem__(payload, "is_cached")
+ if is_cached is not None and type(is_cached) is not bool:
+ return _malformed_result(f"{label} is_cached must be an exact boolean")
+
+ if "cache_key" in payload:
+ cache_key = dict.__getitem__(payload, "cache_key")
+ if cache_key is not None and (
+ type(cache_key) is not str
+ or _bounded_utf8_length(cache_key, _MAX_CACHE_STRING_BYTES) is None
+ ):
+ return _malformed_result(
+ f"{label} cache_key must be a bounded exact string"
+ )
+
+ # ChartData's production schema emits ``cached_dttm``. ``cache_dttm`` was
+ # used by earlier MCP payloads and remains a bounded compatibility alias.
+ # Validate both before cache utilities parse or compare either value.
+ for key in ("cached_dttm", "cache_dttm"):
+ if key not in payload:
+ continue
+ cache_dttm = dict.__getitem__(payload, key)
+ if cache_dttm is None:
+ continue
+ if type(cache_dttm) is str:
+ if _bounded_utf8_length(cache_dttm, _MAX_CACHE_STRING_BYTES) is not None:
+ continue
+ return _malformed_result(f"{label} {key} must be a bounded exact string")
+ if type(cache_dttm) is datetime:
+ tzinfo = cache_dttm.tzinfo
+ if tzinfo is None or any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ continue
+ return _malformed_result(f"{label} {key} has an unsupported timezone")
+ return _malformed_result(
+ f"{label} {key} must be a bounded exact string or datetime"
+ )
+ return None
+
+
+def _normalize_row_value( # noqa: C901
+ value: Any,
+ budget: _ResultBudget,
+ *,
+ temporal_json_numbers: bool = False,
+ preserve_nonfinite_floats: bool = False,
+) -> str | None:
+ """Normalize trusted scalars and return a bounded serialization failure.
+
+ ChartDataCommand materializes a real DataFrame with ``to_dict(records)``, so
+ exact pandas and NumPy scalars can remain in an otherwise valid result. They
+ are converted in place to their canonical JSON-facing builtin values here.
+ Exact containers are inspected through builtin operations only; arbitrary
+ subclasses and scalar hooks remain rejected.
+ """
+ stack: list[
+ tuple[
+ Any,
+ list[Any] | dict[str, Any] | None,
+ int | str | None,
+ int,
+ bool,
+ ]
+ ] = [(value, None, None, 0, False)]
+ active_containers: set[int] = set()
+
+ while stack:
+ item, parent, slot, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "exceeds the nesting depth limit"
+
+ if type(item) is list:
Review Comment:
A ClickHouse `Tuple(Int32, String)` cell arrives as a Python tuple and survives `df_to_records`, but this walker sends `(1, "a")` to unsupported-scalar rejection, so successful queries now fail with `MalformedQueryResult` instead of returning the previously supported JSON array. Could exact builtin tuples be normalized as bounded containers too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4189472695
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,795 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Retain final output sorts and rebind inherited metric expressions."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs = {
+ label
+ for column in new_form_data.get("groupby") or []
+ if (label := _column_label(column)) is not None
+ }
+ metrics = new_form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [new_form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ outputs.update(metric_outputs)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if isinstance(label, str) and label not in outputs:
Review Comment:
Agreed: native ordering can rank by an undisplayed saved metric (metrics_by_name in get_sqla_query). Fixed in edbce563ed (https://github.com/apache/superset/pull/43770/commits/edbce563ed): _orderby_for_final_output_roles only drops a sort whose target was a previous Bullet output role that the final state no longer has; independent sorters such as ["profit", false] are kept. Existing tests for removed dimensions/metrics still pass. Regression test: test_saved_bullet_labels_update_keeps_independent_ranking_metric (row_limit 1).
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -154,6 +168,17 @@ def _compute_effective_force(request: GetChartDataRequest) -> bool:
return request.force_refresh or not request.use_cache
+def _download_form_data(form_data: dict[str, Any], request: Any) -> dict[str, Any]:
+ """Return form_data without UI page sizing for CSV/Excel downloads.
+
+ Server-paginated tables size a query to the UI page; a download keeps the
+ caller's limit instead of shrinking to that page.
+ """
+ if request.format in {"csv", "excel"} and form_data.get("server_pagination"):
Review Comment:
Agreed, the bypass was unguarded. Added in edbce563ed (https://github.com/apache/superset/pull/43770/commits/edbce563ed): test_server_paginated_table_export_queries_full_limit runs get_chart_data through the real build_query_dicts_from_form_data on the saved-fallback, saved+form_data_key and form_data_key-only paths for CSV and Excel, with a command that returns as many rows as the main query LIMIT. It asserts the main query row_limit is 1000 and that 1000 (> 25) rows are exported; with _download_form_data reduced to a no-op all six cases fail.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,610 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if type(value) in {list, tuple, dict}:
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(
+ {"queries": [{"data": [{"value": value}]}]}, temporal_json_numbers=True
+ )
+ if failure is not None or data is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} "
+ "has an invalid or unbounded container value"
+ )
+ normalized = data[0][0]["value"]
+ reason = None
+ elif _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if value_type in {list, dict}:
+ text = _bullet_container_category_text(normalized, dimension, row_index)
+ elif normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_container_category_text(value: Any, dimension: str, row_index: int) -> str:
+ """Stringify validated containers like JavaScript with a bounded text budget."""
+ from superset.mcp_service.chart.query_result import _bounded_utf8_length
+
+ if type(value) is dict:
+ return "[object Object]"
+ parts: list[str] = []
+ size = 0
+ for index, item in enumerate(value):
+ if type(item) in {list, dict}:
+ text = _bullet_container_category_text(item, dimension, row_index)
+ else:
+ text = (
+ ""
+ if item is None
+ else _bullet_category_value(item, dimension, row_index)[1]
+ )
+ text_size = _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES)
+ size += (text_size if text_size is not None else _MAX_BULLET_TEXT_BYTES + 1) + (
+ index > 0
+ )
+ if size > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ parts.append(text)
+ return ",".join(parts)
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ from superset.utils.number_format import D3_FORMAT_RE
+
+ # Specifier length does not bound its requested output precision. Check the
+ # parsed precision before the formatter can allocate or round any value.
+ match = D3_FORMAT_RE.match(format_)
+ if match and match.group(8) is not None and int(match.group(8)) > 20:
+ raise BulletOutputError(
+ "Bullet number format precision must not exceed 20",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet number format {format_!r} is unsupported by previews",
+ error_type="UnsupportedFormat",
+ ) from ex
+ return format_
+
+
+def _format_bullet_number(format_: str, value: float) -> str:
+ """Format finite Bullet values, including the full binary-float range."""
+ from superset.utils.number_format import format_numeric
+
+ try:
+ # Binary64 has at most 309 integer digits. Leave room for the bounded
+ # fractional precision, percent scaling, and a rounding carry without
+ # changing the caller's Decimal context.
+ with localcontext() as context:
+ context.prec = max(context.prec, 334)
+ return format_numeric(format_, value)
+ except OverflowError:
+ # SMART_NUMBER's significant-digit rounding can overflow a finite float
+ # near DBL_MAX. Scientific repr remains deterministic and informative.
+ if format_ in {"SMART_NUMBER", "SMART_NUMBER_SIGNED"} and math.isfinite(value):
+ prefix = "+" if format_ == "SMART_NUMBER_SIGNED" and value > 0 else ""
+ return prefix + repr(value)
+ raise
+
+
+def _containing_bullet_range_label(
+ measure: float, ranges: list[float], labels: list[str]
+) -> str | None:
+ """Match the frontend's labelled containing-range tooltip selection."""
+ ascending = sorted(
+ (
+ (value, labels[index] if index < len(labels) else "")
+ for index, value in enumerate(ranges)
+ ),
+ key=lambda entry: entry[0],
+ )
+ for threshold, label in ascending:
+ if measure <= threshold:
+ return label or None
+ if ascending and ascending[-1][1]:
+ return f"> {ascending[-1][1]}"
+ return None
+
+
+def resolve_bullet_render_model( # noqa: C901
+ data: List[Dict[str, Any]],
+ form_data: Dict[str, Any],
+ *,
+ validate_format: bool = True,
+) -> BulletRenderModel:
+ """Resolve Bullet rows; check preview formatter support only when rendering."""
+ if type(data) is not list:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ for row_index in range(list.__len__(data)):
+ row = list.__getitem__(data, row_index)
+ if type(row) is not dict:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ if dict.__len__(row) > _MAX_BULLET_FIELDS:
+ raise BulletOutputError("Bullet query row exceeds the field limit")
+ for key in dict.keys(row):
+ if type(key) is not str:
+ raise BulletOutputError("Bullet query row keys must be strings")
+ if len(key) > _MAX_BULLET_FIELD_BYTES:
+ raise BulletOutputError("Bullet query row key exceeds the size limit")
+
+ if type(form_data) is not dict:
+ raise BulletOutputError("Bullet form data must be an object")
+
+ metric_label = _form_metric_label(dict.get(form_data, "metric"))
+ if not metric_label:
+ raise BulletOutputError("Bullet metric has no declared result alias")
+ raw_groupby = dict.get(form_data, "groupby")
+ if raw_groupby is None:
+ raw_groupby = []
+ if type(raw_groupby) is not list:
Review Comment:
Agreed: Bullet/transformProps.ts reads ensureIsArray(groupby).map(getColumnLabel), so a scalar groupby is a one-level hierarchy and an unlabeled SQL dimension is keyed by its sqlExpression. Fixed in edbce563ed (https://github.com/apache/superset/pull/43770/commits/edbce563ed): resolve_bullet_render_model wraps a scalar/object groupby and _form_column_label falls back to sqlExpression (shared by data reads, exports and both previews); the update path normalizes the saved hierarchy the same way (bullet_groupby_list) so a sort on a scalar "Region" resolves instead of iterating characters. Regression tests: test_bullet_saved_groupby_follows_native_ensure_is_array_and_label and test_saved_bullet_scalar_groupby_resolves_sort_update.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,795 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
Review Comment:
Agreed: other plugins keep these through merge_same_viz_form_data, but the Bullet allowlist dropped them. Fixed in edbce563ed (https://github.com/apache/superset/pull/43770/commits/edbce563ed): merge_bullet_form_data preserves extra_form_data and extra_filters, and validate_merged_bullet_form_data excludes them from the typed validation copy (as it does url_params) while the compiled/persisted state keeps them. Regression tests: test_bullet_labels_update_preserves_native_query_context (preview merge + saved preview) and test_saved_bullet_labels_update_persists_native_query_context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-6007432935
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4191487503
##########
superset/mcp_service/chart/compile.py:
##########
@@ -209,6 +240,22 @@ def _adhoc_filter_column_valid(
WHERE filters must reference a physical column; HAVING filters may also
reference a saved metric because Superset resolves metric names there.
"""
+ column_names = [item["name"] for item in dataset_context.available_columns]
+ metric_names = [metric["name"] for metric in dataset_context.available_metrics]
+ if column in column_names or (clause == "HAVING" and column in metric_names):
+ return True
+ try:
+ if resolve_dataset_column(column, dataset_context) is not None:
+ return True
+ except ValueError:
Review Comment:
`AmbiguousDatasetReferenceError` subclasses `ValueError`, so this `except ValueError` swallows it before the caller's `except AmbiguousDatasetReferenceError` can run. With physical columns `Score` and `SCORE` and a preserved WHERE filter on `score`, the user now gets the generic `CHART_VALIDATION_FAILED` "not in dataset" error instead of `AMBIGUOUS_DATASET_REFERENCE` listing both candidates. The earlier `test_preserved_filter_ambiguity_is_actionable` that asserted this was removed in the same change, so nothing catches it. Could the ambiguity be re-raised here (and that regression restored)? The statements after the `return` on the previous lines are also unreachable now.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1738,560 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ # Native extractQueryFields excludes empty-string column references.
+ table_columns = [
+ column
+ for column in _as_list(
+ form_data.get("all_columns") if raw_mode else form_data.get("groupby")
+ )
+ if column != ""
+ ]
+ table_metrics = [] if raw_mode else _as_list(form_data.get("metrics"))
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ query_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ query_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ # Native comparisons use ordinary metrics, before percentage-only metrics
+ # are added to the selected query and contribution operator.
+ has_time_comparison = _time_comparison(form_data, table_metrics)
+ offsets = _table_time_offsets(form_data, {**query, "metrics": table_metrics})
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if has_time_comparison:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if has_time_comparison and offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ # ``query["row_limit"]`` is the normalized caller limit (explicit request
+ # limit or the saved row_limit, which may be stored as a string); page
+ # sizing narrows it but never replaces it.
+ configured_limit = _positive_int(query.get("row_limit"))
+ if form_data.get("server_pagination"):
+ if page_size := _positive_int(form_data.get("server_page_length")):
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if query_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ query_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
+ if metric is None:
+ plural_metrics = _as_list(form_data.get("metrics"))
+ metric = plural_metrics[0] if plural_metrics else None
+ columns = _resolve_big_number_query_columns(form_data) if trendline else []
+ query = build_single_query_dict(
+ form_data,
+ columns,
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if trendline:
+ # Big Number has no series dimension. Its frontend pivot receives
+ # the common base QueryObject (whose columns are empty), not the
+ # final QueryObject after the explicit x-axis is added. Preserve
+ # that distinction instead of falling back to the final columns.
+ query["series_columns"] = []
+ if not form_data.get("x_axis"):
+ query["is_timeseries"] = True
+ query["post_processing"] = _timeseries_post_processing(form_data, query)
+ if form_data.get("aggregation") == "raw":
+ return [
+ query,
+ {
+ **query,
+ "columns": [],
+ "is_timeseries": False,
+ "post_processing": [],
+ },
+ ]
+ return [query]
+
+
+def build_waterfall_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Waterfall buildQuery with raw-axis ordering."""
+ metrics, groupby = resolve_metrics_and_groupby(form_data)
+ # normalizeTimeColumn runs after Waterfall's buildQuery callback. It
+ # wraps only the final x-axis column; orderby deliberately retains the
+ # raw control value produced inside the callback.
+ raw_axis = form_data.get("x_axis") or form_data.get("granularity_sqla")
+ query_axis = (
+ _normalized_x_axis_query_field(form_data)
+ if form_data.get("x_axis")
+ else raw_axis
+ )
+ waterfall_columns = ([query_axis] if query_axis else []) + groupby
+ raw_ordering_columns = ([raw_axis] if raw_axis else []) + groupby
+ query = build_single_query_dict(
+ form_data,
+ waterfall_columns,
+ metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=None,
+ )
+ query["orderby"] = [[column, True] for column in raw_ordering_columns]
+ if form_data.get("x_axis"):
+ query.pop("is_timeseries", None)
+ return [query]
+
+
+def build_mixed_timeseries_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Mixed Timeseries buildQuery for both independent layers."""
+ from superset.utils.core import split_adhoc_filters_into_base_filters
+
+ queries: list[dict[str, Any]] = []
+ x_axis = _normalized_x_axis_query_field(form_data)
+ for secondary in (False, True):
+ layer = _mixed_layer_form_data(form_data, secondary=secondary)
+ if secondary and form_data.get("adhoc_filters_b") is not None:
+ for key in ("filters", "where", "having"):
+ layer.pop(key, None)
+ secondary_filters = list(form_data.get("adhoc_filters_b") or [])
+ # Request-level filters (extra_form_data / extra_filters) are not
+ # suffixed, so they apply to both layers on top of the layer's own.
+ secondary_filters.extend(
+ filter_
+ for filter_ in form_data.get("adhoc_filters") or []
+ if isinstance(filter_, dict)
+ and filter_.get("isExtra")
+ and filter_ not in secondary_filters
+ )
+ layer["adhoc_filters"] = secondary_filters
+ split_adhoc_filters_into_base_filters(layer, engine)
+ layer_metrics = _timeseries_base_metrics(layer)
+ layer_groupby = _as_list(layer.get("groupby"))
+ columns = [*(_as_list(x_axis) if x_axis else []), *layer_groupby]
+ query = build_single_query_dict(
+ layer,
+ _dedupe_query_fields(columns, _column_label),
+ layer_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=layer.get("orderby"),
+ )
+ query["series_columns"] = layer_groupby
+ if not x_axis:
+ query["is_timeseries"] = True
+ comparison = _time_comparison(layer, layer_metrics)
+ query["time_offsets"] = (
+ _as_list(layer.get("time_compare")) if comparison else []
+ )
+ query["post_processing"] = _timeseries_post_processing(
+ layer,
+ query,
+ operator_metrics=layer_metrics,
+ )
+ _normalize_orderby(query)
+ queries.append(query)
+ return queries
+
+
+def build_timeseries_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render ECharts Timeseries buildQuery and its post-processing."""
+ x_axis = _normalized_x_axis_query_field(form_data)
+ timeseries_metrics = _timeseries_base_metrics(form_data)
+ extra_metrics = _timeseries_extra_metrics(form_data)
+ timeseries_groupby = _as_list(form_data.get("groupby"))
+ columns = [*(_as_list(x_axis) if x_axis else []), *timeseries_groupby]
+ query = build_single_query_dict(
+ form_data,
+ _dedupe_query_fields(columns, _column_label),
+ [*timeseries_metrics, *extra_metrics],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ query["series_columns"] = timeseries_groupby
+ if not x_axis:
+ query["is_timeseries"] = True
+ comparison = _time_comparison(form_data, timeseries_metrics)
+ query["time_offsets"] = (
+ _as_list(form_data.get("time_compare")) if comparison else []
+ )
+ query["time_compare_full_range"] = bool(
+ query["time_offsets"] and form_data.get("time_compare_full_range")
+ )
+ query["post_processing"] = _timeseries_post_processing(
Review Comment:
This pivot post-processing turns a grouped timeseries into wide columns such as `SUM(revenue), EU` and `SUM(revenue), US`, but the generic Vega-Lite renderer in `preview_utils.py` still looks for the long-form `region` column plus one metric column. For a preview of a line chart with `x=event_date`, `SUM(revenue)` and `groupby=["region"]`, `_resolve_y_metric_column` falls back to the first numeric column and no color encoding is emitted, so only one series is drawn and the preview still reports success. Could the preview path unpivot these rows (or the XY plugin supply a matching preview adapter), with a grouped-series test?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1738,560 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ # Native extractQueryFields excludes empty-string column references.
+ table_columns = [
+ column
+ for column in _as_list(
+ form_data.get("all_columns") if raw_mode else form_data.get("groupby")
+ )
+ if column != ""
+ ]
+ table_metrics = [] if raw_mode else _as_list(form_data.get("metrics"))
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ query_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ query_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ # Native comparisons use ordinary metrics, before percentage-only metrics
+ # are added to the selected query and contribution operator.
+ has_time_comparison = _time_comparison(form_data, table_metrics)
+ offsets = _table_time_offsets(form_data, {**query, "metrics": table_metrics})
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if has_time_comparison:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if has_time_comparison and offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ # ``query["row_limit"]`` is the normalized caller limit (explicit request
+ # limit or the saved row_limit, which may be stored as a string); page
+ # sizing narrows it but never replaces it.
+ configured_limit = _positive_int(query.get("row_limit"))
+ if form_data.get("server_pagination"):
+ if page_size := _positive_int(form_data.get("server_page_length")):
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if query_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ query_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
+ if metric is None:
+ plural_metrics = _as_list(form_data.get("metrics"))
+ metric = plural_metrics[0] if plural_metrics else None
+ columns = _resolve_big_number_query_columns(form_data) if trendline else []
+ query = build_single_query_dict(
+ form_data,
+ columns,
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if trendline:
+ # Big Number has no series dimension. Its frontend pivot receives
+ # the common base QueryObject (whose columns are empty), not the
+ # final QueryObject after the explicit x-axis is added. Preserve
+ # that distinction instead of falling back to the final columns.
+ query["series_columns"] = []
+ if not form_data.get("x_axis"):
+ query["is_timeseries"] = True
+ query["post_processing"] = _timeseries_post_processing(form_data, query)
+ if form_data.get("aggregation") == "raw":
+ return [
+ query,
+ {
+ **query,
+ "columns": [],
+ "is_timeseries": False,
+ "post_processing": [],
+ },
+ ]
+ return [query]
+
+
+def build_waterfall_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Waterfall buildQuery with raw-axis ordering."""
+ metrics, groupby = resolve_metrics_and_groupby(form_data)
+ # normalizeTimeColumn runs after Waterfall's buildQuery callback. It
+ # wraps only the final x-axis column; orderby deliberately retains the
+ # raw control value produced inside the callback.
+ raw_axis = form_data.get("x_axis") or form_data.get("granularity_sqla")
+ query_axis = (
+ _normalized_x_axis_query_field(form_data)
+ if form_data.get("x_axis")
+ else raw_axis
+ )
+ waterfall_columns = ([query_axis] if query_axis else []) + groupby
+ raw_ordering_columns = ([raw_axis] if raw_axis else []) + groupby
+ query = build_single_query_dict(
+ form_data,
+ waterfall_columns,
+ metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=None,
+ )
+ query["orderby"] = [[column, True] for column in raw_ordering_columns]
+ if form_data.get("x_axis"):
+ query.pop("is_timeseries", None)
+ return [query]
+
+
+def build_mixed_timeseries_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Mixed Timeseries buildQuery for both independent layers."""
+ from superset.utils.core import split_adhoc_filters_into_base_filters
+
+ queries: list[dict[str, Any]] = []
+ x_axis = _normalized_x_axis_query_field(form_data)
+ for secondary in (False, True):
+ layer = _mixed_layer_form_data(form_data, secondary=secondary)
+ if secondary and form_data.get("adhoc_filters_b") is not None:
+ for key in ("filters", "where", "having"):
+ layer.pop(key, None)
+ secondary_filters = list(form_data.get("adhoc_filters_b") or [])
+ # Request-level filters (extra_form_data / extra_filters) are not
+ # suffixed, so they apply to both layers on top of the layer's own.
+ secondary_filters.extend(
+ filter_
+ for filter_ in form_data.get("adhoc_filters") or []
+ if isinstance(filter_, dict)
+ and filter_.get("isExtra")
+ and filter_ not in secondary_filters
+ )
+ layer["adhoc_filters"] = secondary_filters
+ split_adhoc_filters_into_base_filters(layer, engine)
+ layer_metrics = _timeseries_base_metrics(layer)
+ layer_groupby = _as_list(layer.get("groupby"))
+ columns = [*(_as_list(x_axis) if x_axis else []), *layer_groupby]
+ query = build_single_query_dict(
+ layer,
+ _dedupe_query_fields(columns, _column_label),
+ layer_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
Review Comment:
Passing the shared `order_desc` here overrides the secondary layer's own `order_desc_b` that `_mixed_layer_form_data` just folded into `layer`. Callers such as `get_chart_preview` and the cached-form-data path in `get_chart_data` always pass `form_data.get("order_desc", True)`, so a Mixed chart with `order_desc=False`, `order_desc_b=True`, `limit_b=1` and `timeseries_limit_metric_b="costs"` picks the lowest-cost series for layer B instead of the highest, which changes which data comes back rather than just its order. Could the secondary layer prefer its own resolved `order_desc` when a layer-specific value exists, with a regression that passes the primary flag explicitly?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,2010 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+_EXCEL_MIN_DATE = date(1900, 1, 1)
+# Leave room for XLSX readers' millisecond rounding at the final serial day.
+_EXCEL_MAX_TIME = time(23, 59, 59, 999000)
+_EXCEL_MAX_DATETIME = datetime.combine(date.max, _EXCEL_MAX_TIME)
+# Serial day 2958465 is 9999-12-31 in Excel's 1900 date system.
+_EXCEL_MAX_DURATION = timedelta(
+ days=2958465, hours=23, minutes=59, seconds=59, milliseconds=999
+)
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
return None
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget, *, row_shaped: bool = False
+) -> str | None:
+ """Bound metadata, charging row-shaped indexes to the row-data work budget."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if row_shaped:
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "metadata contains too many total values"
+ else:
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list or type(item) is tuple:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = len(item)
+ # Chart Data emits one indexname per row. Only that outer array
+ # receives the row limit; containers within an index stay bounded.
+ max_items = (
+ MAX_QUERY_RESULT_ROWS
+ if row_shaped and depth == 0
+ else _MAX_ROW_CONTAINER_ITEMS
+ )
+ if width > max_items:
+ return "metadata contains an oversized array"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(width, mapping=False),
+ metadata=True,
+ ):
+ return reason
+ stack.extend((item[index], depth + 1, False) for index in range(width))
+ continue
+
+ normalized, reason = _normalize_trusted_scalar(
+ item, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ )
+ if reason is not None:
+ return f"metadata {reason}"
+ if reason := _charge_json_bytes(
+ budget,
+ _normalized_scalar_json_size(
+ normalized, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ ),
+ metadata=True,
+ ):
+ return reason
+ return None
+
+
+def _metadata_failure( # noqa: C901
+ payload: dict[str, Any], label: str
+) -> ChartError | None:
+ """Validate bounded cache and row-count metadata before any consumer."""
+ for key in ("rowcount", "total_rows"):
+ if key in payload and dict.__getitem__(payload, key) is not None:
+ try:
+ bounded_result_row_count(dict.__getitem__(payload, key))
+ except ValueError as ex:
+ return _malformed_result(f"{label} {key} {ex}")
+
+ if "is_cached" in payload:
+ is_cached = dict.__getitem__(payload, "is_cached")
+ if is_cached is not None and type(is_cached) is not bool:
+ return _malformed_result(f"{label} is_cached must be an exact boolean")
+
+ if "cache_key" in payload:
+ cache_key = dict.__getitem__(payload, "cache_key")
+ if cache_key is not None and (
+ type(cache_key) is not str
+ or _bounded_utf8_length(cache_key, _MAX_CACHE_STRING_BYTES) is None
+ ):
+ return _malformed_result(
+ f"{label} cache_key must be a bounded exact string"
+ )
+
+ # ChartData's production schema emits ``cached_dttm``. ``cache_dttm`` was
+ # used by earlier MCP payloads and remains a bounded compatibility alias.
+ # Validate both before cache utilities parse or compare either value.
+ for key in ("cached_dttm", "cache_dttm"):
+ if key not in payload:
+ continue
+ cache_dttm = dict.__getitem__(payload, key)
+ if cache_dttm is None:
+ continue
+ if type(cache_dttm) is str:
+ if _bounded_utf8_length(cache_dttm, _MAX_CACHE_STRING_BYTES) is not None:
+ continue
+ return _malformed_result(f"{label} {key} must be a bounded exact string")
+ if type(cache_dttm) is datetime:
+ tzinfo = cache_dttm.tzinfo
+ if tzinfo is None or any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ continue
+ return _malformed_result(f"{label} {key} has an unsupported timezone")
+ return _malformed_result(
+ f"{label} {key} must be a bounded exact string or datetime"
+ )
+ return None
+
+
+def _excel_temporal_cell(value: Any) -> Any | None:
+ """Return a naive builtin temporal value an XLSX writer stores natively.
+
+ Called only after ``_normalize_trusted_scalar`` accepted ``value``. XLSX has
+ no timezone-aware date cells, so aware values keep their ISO text, as do
+ NumPy durations whose unit may be ambiguous. Values outside the 1900 date
+ system's range, including negative durations, also keep their ISO text.
+ """
+ value_type = type(value)
+ if value_type is date:
+ return value if value >= _EXCEL_MIN_DATE else None
+ if value_type is timedelta:
+ return value if timedelta(0) <= value <= _EXCEL_MAX_DURATION else None
+ if value_type is datetime:
+ return (
+ value
+ if value.tzinfo is None
+ and value.date() >= _EXCEL_MIN_DATE
+ and value <= _EXCEL_MAX_DATETIME
+ else None
+ )
+ if value_type is time:
+ return value if value.tzinfo is None and value <= _EXCEL_MAX_TIME else None
+ if value_type is np.datetime64:
+ value = pd.Timestamp(value)
+ value_type = pd.Timestamp
+ if value_type is pd.Timestamp:
+ if value.tzinfo is not None or not 1900 <= value.year <= 9999:
+ return None
+ # Excel stores millisecond precision; drop nanoseconds explicitly
+ # instead of through pandas' lossy-conversion warning.
+ return _excel_temporal_cell(
+ datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ )
+ if value_type is pd.Timedelta:
+ if not 0 <= value.days <= _EXCEL_MAX_DURATION.days:
+ return None
+ return _excel_temporal_cell(
+ timedelta(
+ days=value.days, seconds=value.seconds, microseconds=value.microseconds
+ )
+ )
+ return None
+
+
+def _normalize_row_value( # noqa: C901
+ value: Any,
+ budget: _ResultBudget,
+ *,
+ temporal_json_numbers: bool = False,
+ preserve_nonfinite_floats: bool = False,
+ preserve_excel_temporals: bool = False,
+) -> str | None:
+ """Normalize trusted scalars and return a bounded serialization failure.
+
+ ChartDataCommand materializes a real DataFrame with ``to_dict(records)``, so
+ exact pandas and NumPy scalars can remain in an otherwise valid result. They
+ are converted in place to their canonical JSON-facing builtin values here.
+ Exact containers are inspected through builtin operations only; arbitrary
+ subclasses and scalar hooks remain rejected. ``preserve_excel_temporals``
+ keeps naive row-level date, time, datetime, and duration cells as builtin
+ temporal objects for XLSX exports; size budgets still use their JSON text.
+ """
+ stack: list[
+ tuple[
+ Any,
+ list[Any] | dict[str, Any] | None,
+ int | str | None,
+ int,
+ bool,
+ ]
+ ] = [(value, None, None, 0, False)]
+ active_containers: set[int] = set()
+
+ while stack:
+ item, parent, slot, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "exceeds the nesting depth limit"
+
+ if type(item) is np.ndarray:
+ # Arrow list cells are exact arrays. Expand one bounded dimension at
+ # a time, never calling tolist() before shape/work checks or trusting
+ # ndarray subclasses. Object-array children use the same scalar and
+ # cycle guards as builtin containers.
+ identity = id(item)
+ if identity in active_containers:
+ return "contains cyclic containers"
+ if depth + item.ndim > _MAX_ROW_CONTAINER_DEPTH:
+ return "exceeds the nesting depth limit"
+ if any(width > _MAX_ROW_CONTAINER_ITEMS for width in item.shape):
+ return "contains an oversized array"
+ if item.size > MAX_QUERY_RESULT_VALUES - budget.values:
+ return "exceeds the total value limit"
+ active_containers.add(identity)
+ stack.append((item, None, None, depth, True))
+ scalar_array = item.ndim == 0
+ item = (
+ np.ndarray.__getitem__(item, ())
+ if scalar_array
+ else [
+ np.ndarray.__getitem__(item, index)
+ for index in range(item.shape[0])
+ ]
+ )
+ if type(parent) is list:
+ assert type(slot) is int
+ list.__setitem__(parent, slot, item)
+ elif type(parent) is dict:
+ assert type(slot) is str
+ dict.__setitem__(parent, slot, item)
+ if scalar_array:
+ stack.append((item, parent, slot, depth + 1, False))
+ continue
+
+ if type(item) is list or type(item) is tuple:
+ identity = id(item)
+ if identity in active_containers:
+ return "contains cyclic containers"
+ width = len(item)
+ if width > _MAX_ROW_CONTAINER_ITEMS:
+ return "contains an oversized array"
+ active_containers.add(identity)
+ stack.append((item, None, None, depth, True))
+ if type(item) is tuple:
+ item = list(item)
+ if type(parent) is list:
+ assert type(slot) is int
+ list.__setitem__(parent, slot, item)
+ elif type(parent) is dict:
+ assert type(slot) is str
+ dict.__setitem__(parent, slot, item)
+ if reason := _add_result_value_to_budget(item, budget):
+ return reason
+ if reason := _charge_json_bytes(
+ budget, _container_json_syntax_size(width, mapping=False)
+ ):
+ return reason
+ stack.extend(
+ (list.__getitem__(item, index), item, index, depth + 1, False)
+ for index in range(width)
+ )
+ continue
+
+ if type(item) is dict:
+ if reason := _add_result_value_to_budget(item, budget):
+ return reason
+ identity = id(item)
+ if identity in active_containers:
+ return "contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, None, None, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget, _container_json_syntax_size(item_count, mapping=True)
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size):
+ return reason
+ stack.append((child, item, key, depth + 1, False))
+ continue
+
+ normalized: Any
+ generic_scalar = False
+ if (
+ preserve_nonfinite_floats
+ and type(item) is float
+ and not math.isfinite(item)
+ ):
+ normalized, reason = item, None
+ elif temporal_json_numbers and _is_chart_data_temporal_scalar(item):
+ normalized, reason = _chart_data_temporal_number(item)
+ elif temporal_json_numbers and _is_chart_data_duration_scalar(item):
+ normalized, reason = _chart_data_duration_text(item)
+ else:
+ normalized, reason = _normalize_trusted_scalar(item)
+ generic_scalar = True
+ if reason is not None:
+ return reason
+ if reason := _add_result_value_to_budget(normalized, budget):
+ return reason
+ replacement = normalized
+ # Plugins with Chart Data temporal wire semantics (numbers/duration
+ # text) keep that projection in every export format.
+ if (
+ preserve_excel_temporals
Review Comment:
The Excel-only preservation here means shared normalization still rewrites temporal cells for every other format, so CSV exports of existing non-Bullet charts change from `2026-10-05 12:00:00` to `2026-10-05T12:00:00` and a duration from `1:30:00` to `PT1H30M`. A downstream job that parses the old CSV text with `strptime("%Y-%m-%d %H:%M:%S")` would start failing after upgrade with no chart change, and `UPDATING.md` doesn't mention it. Should CSV keep the established formatting, or is this change intended and worth documenting?
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1349,6 +1359,111 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Inherited native SQL dimensions remain query expressions when updating metrics
+or `order_by`; reference their output labels to sort by them. Caller-supplied
+`dimensions` must still be physical columns, not SQL expressions.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling. Bullet metric strings use
+JavaScript numeric spellings: underscore separators and non-ASCII digits are
+rejected rather than interpreted as numbers. Bounded array-valued dimensions
+retain their raw values in data reads and exports; preview category labels use
+JavaScript string conversion (for example, `[1, 2]` displays as `1,2`). Native
+SQL metrics without a label use their SQL expression as the output label.
+
+Presentation-only updates preserve saved predicates, including legacy top-level
+`where`, `having`, and `filters`, and normalize the native `order_by_cols` alias.
+When both `orderby` and `order_by_cols` are saved, their sort entries are
+concatenated in form-data key order, including when `orderby` is empty.
+Explicit `filters: []` and `order_by: []` clear those inherited controls.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+Labels that become empty after sanitization retain their slots so later labels
+remain aligned with their corresponding values.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates. Omitted label controls preserve the saved state only when their
+corresponding value controls (`ranges`, `markers`, or `marker_lines`) are also
+omitted. Replacing a value control without its labels clears the saved labels;
+resend the labels to retain annotations with the replacement values.
+
+Saved native `ranges`, `markers`, and `marker_lines` controls ignore empty,
+non-numeric, and NaN tokens, matching Explore. If no numeric range remains, the
+preview uses the default band up to 110% of the largest measure. Unrelated
+updates preserve these saved controls. Newly authored typed lists must contain
+finite numbers; infinite native values and oversized controls remain errors.
+
+On same-dataset chart updates, `filters: []` clears both user filters and the
+generated dashboard-time binding. `temporal_column: null` clears only that
+generated binding, preserving user filters. Omitting those controls preserves
+the saved binding. Bullet Vega previews keep separate indexed rows even when
+dimension display labels are identical, and support both `SMART_NUMBER` and
+`SMART_NUMBER_SIGNED` number formats. Bullet numeric format precision is limited
+to 20 digits before formatting, including on data reads.
+
+Native Bullet temporal filters retain the active filter's subject and range as a
+pair; `No filter` placeholders do not supply the subject of another active range.
+Typed configs support one such pair. Multiple active native temporal filters, or
+conflicts with explicit `temporal_column`/`time_range`, are rejected rather than
+silently dropping or moving a predicate.
+
+### Query result limits
+
+MCP chart tools accept up to 50,000 rows per query. The row-shaped `indexnames`
+array emitted by Chart Data uses the same 50,000-entry limit, rather than the
+4,096-entry limit for other metadata arrays. Index entries count toward the
+shared row-data work budget and retain the 1 MiB aggregate metadata byte limit.
+Multi-dimension pivot index tuples are serialized as JSON arrays. Nested
+containers within an index entry retain the standard container limits.
+
+The shared query-result validator also enforces **non-configurable hard limits**:
+
+- **64 KiB (65,536 UTF-8 bytes) per text cell/string value**. A single
Review Comment:
These limits read broader than the code enforces. The 64 KiB cap applies to row cells, while metadata strings fall under the separate metadata budget (`test_query_result_accepts_full_sql_above_source_cell_string_limit` accepts a 70 KiB SQL string), and the 20-digit format-precision check at the top of this section only runs when preview validation is on, so a saved Bullet with `y_axis_format: ".21e"` still returns raw data. An operator could rewrite a working saved chart or query unnecessarily; could the wording say these apply to row cells and to previews respectively (same phrasing in `UPDATING.md`)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4191859835
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1738,560 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ # Native extractQueryFields excludes empty-string column references.
+ table_columns = [
+ column
+ for column in _as_list(
+ form_data.get("all_columns") if raw_mode else form_data.get("groupby")
+ )
+ if column != ""
+ ]
+ table_metrics = [] if raw_mode else _as_list(form_data.get("metrics"))
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ query_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ query_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ # Native comparisons use ordinary metrics, before percentage-only metrics
+ # are added to the selected query and contribution operator.
+ has_time_comparison = _time_comparison(form_data, table_metrics)
+ offsets = _table_time_offsets(form_data, {**query, "metrics": table_metrics})
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if has_time_comparison:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if has_time_comparison and offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ # ``query["row_limit"]`` is the normalized caller limit (explicit request
+ # limit or the saved row_limit, which may be stored as a string); page
+ # sizing narrows it but never replaces it.
+ configured_limit = _positive_int(query.get("row_limit"))
+ if form_data.get("server_pagination"):
+ if page_size := _positive_int(form_data.get("server_page_length")):
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if query_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ query_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
+ if metric is None:
+ plural_metrics = _as_list(form_data.get("metrics"))
+ metric = plural_metrics[0] if plural_metrics else None
+ columns = _resolve_big_number_query_columns(form_data) if trendline else []
+ query = build_single_query_dict(
+ form_data,
+ columns,
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if trendline:
+ # Big Number has no series dimension. Its frontend pivot receives
+ # the common base QueryObject (whose columns are empty), not the
+ # final QueryObject after the explicit x-axis is added. Preserve
+ # that distinction instead of falling back to the final columns.
+ query["series_columns"] = []
+ if not form_data.get("x_axis"):
+ query["is_timeseries"] = True
+ query["post_processing"] = _timeseries_post_processing(form_data, query)
+ if form_data.get("aggregation") == "raw":
+ return [
+ query,
+ {
+ **query,
+ "columns": [],
+ "is_timeseries": False,
+ "post_processing": [],
+ },
+ ]
+ return [query]
+
+
+def build_waterfall_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Waterfall buildQuery with raw-axis ordering."""
+ metrics, groupby = resolve_metrics_and_groupby(form_data)
+ # normalizeTimeColumn runs after Waterfall's buildQuery callback. It
+ # wraps only the final x-axis column; orderby deliberately retains the
+ # raw control value produced inside the callback.
+ raw_axis = form_data.get("x_axis") or form_data.get("granularity_sqla")
+ query_axis = (
+ _normalized_x_axis_query_field(form_data)
+ if form_data.get("x_axis")
+ else raw_axis
+ )
+ waterfall_columns = ([query_axis] if query_axis else []) + groupby
+ raw_ordering_columns = ([raw_axis] if raw_axis else []) + groupby
+ query = build_single_query_dict(
+ form_data,
+ waterfall_columns,
+ metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=None,
+ )
+ query["orderby"] = [[column, True] for column in raw_ordering_columns]
+ if form_data.get("x_axis"):
+ query.pop("is_timeseries", None)
+ return [query]
+
+
+def build_mixed_timeseries_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Mixed Timeseries buildQuery for both independent layers."""
+ from superset.utils.core import split_adhoc_filters_into_base_filters
+
+ queries: list[dict[str, Any]] = []
+ x_axis = _normalized_x_axis_query_field(form_data)
+ for secondary in (False, True):
+ layer = _mixed_layer_form_data(form_data, secondary=secondary)
+ if secondary and form_data.get("adhoc_filters_b") is not None:
+ for key in ("filters", "where", "having"):
+ layer.pop(key, None)
+ secondary_filters = list(form_data.get("adhoc_filters_b") or [])
+ # Request-level filters (extra_form_data / extra_filters) are not
+ # suffixed, so they apply to both layers on top of the layer's own.
+ secondary_filters.extend(
+ filter_
+ for filter_ in form_data.get("adhoc_filters") or []
+ if isinstance(filter_, dict)
+ and filter_.get("isExtra")
+ and filter_ not in secondary_filters
+ )
+ layer["adhoc_filters"] = secondary_filters
+ split_adhoc_filters_into_base_filters(layer, engine)
+ layer_metrics = _timeseries_base_metrics(layer)
+ layer_groupby = _as_list(layer.get("groupby"))
+ columns = [*(_as_list(x_axis) if x_axis else []), *layer_groupby]
+ query = build_single_query_dict(
+ layer,
+ _dedupe_query_fields(columns, _column_label),
+ layer_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
Review Comment:
Fixed in 4f9f6c1a814d57210b46fbfd479ec9f108f83528. Layer B now prefers its explicit order_desc_b over the primary direction passed by the caller. The regression passes order_desc=False explicitly and verifies the secondary series limit, ranking metric, and both direction overrides.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1738,560 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ # Native extractQueryFields excludes empty-string column references.
+ table_columns = [
+ column
+ for column in _as_list(
+ form_data.get("all_columns") if raw_mode else form_data.get("groupby")
+ )
+ if column != ""
+ ]
+ table_metrics = [] if raw_mode else _as_list(form_data.get("metrics"))
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ query_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ query_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ # Native comparisons use ordinary metrics, before percentage-only metrics
+ # are added to the selected query and contribution operator.
+ has_time_comparison = _time_comparison(form_data, table_metrics)
+ offsets = _table_time_offsets(form_data, {**query, "metrics": table_metrics})
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if has_time_comparison:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if has_time_comparison and offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ # ``query["row_limit"]`` is the normalized caller limit (explicit request
+ # limit or the saved row_limit, which may be stored as a string); page
+ # sizing narrows it but never replaces it.
+ configured_limit = _positive_int(query.get("row_limit"))
+ if form_data.get("server_pagination"):
+ if page_size := _positive_int(form_data.get("server_page_length")):
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if query_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ query_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
+ if metric is None:
+ plural_metrics = _as_list(form_data.get("metrics"))
+ metric = plural_metrics[0] if plural_metrics else None
+ columns = _resolve_big_number_query_columns(form_data) if trendline else []
+ query = build_single_query_dict(
+ form_data,
+ columns,
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if trendline:
+ # Big Number has no series dimension. Its frontend pivot receives
+ # the common base QueryObject (whose columns are empty), not the
+ # final QueryObject after the explicit x-axis is added. Preserve
+ # that distinction instead of falling back to the final columns.
+ query["series_columns"] = []
+ if not form_data.get("x_axis"):
+ query["is_timeseries"] = True
+ query["post_processing"] = _timeseries_post_processing(form_data, query)
+ if form_data.get("aggregation") == "raw":
+ return [
+ query,
+ {
+ **query,
+ "columns": [],
+ "is_timeseries": False,
+ "post_processing": [],
+ },
+ ]
+ return [query]
+
+
+def build_waterfall_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Waterfall buildQuery with raw-axis ordering."""
+ metrics, groupby = resolve_metrics_and_groupby(form_data)
+ # normalizeTimeColumn runs after Waterfall's buildQuery callback. It
+ # wraps only the final x-axis column; orderby deliberately retains the
+ # raw control value produced inside the callback.
+ raw_axis = form_data.get("x_axis") or form_data.get("granularity_sqla")
+ query_axis = (
+ _normalized_x_axis_query_field(form_data)
+ if form_data.get("x_axis")
+ else raw_axis
+ )
+ waterfall_columns = ([query_axis] if query_axis else []) + groupby
+ raw_ordering_columns = ([raw_axis] if raw_axis else []) + groupby
+ query = build_single_query_dict(
+ form_data,
+ waterfall_columns,
+ metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=None,
+ )
+ query["orderby"] = [[column, True] for column in raw_ordering_columns]
+ if form_data.get("x_axis"):
+ query.pop("is_timeseries", None)
+ return [query]
+
+
+def build_mixed_timeseries_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Mixed Timeseries buildQuery for both independent layers."""
+ from superset.utils.core import split_adhoc_filters_into_base_filters
+
+ queries: list[dict[str, Any]] = []
+ x_axis = _normalized_x_axis_query_field(form_data)
+ for secondary in (False, True):
+ layer = _mixed_layer_form_data(form_data, secondary=secondary)
+ if secondary and form_data.get("adhoc_filters_b") is not None:
+ for key in ("filters", "where", "having"):
+ layer.pop(key, None)
+ secondary_filters = list(form_data.get("adhoc_filters_b") or [])
+ # Request-level filters (extra_form_data / extra_filters) are not
+ # suffixed, so they apply to both layers on top of the layer's own.
+ secondary_filters.extend(
+ filter_
+ for filter_ in form_data.get("adhoc_filters") or []
+ if isinstance(filter_, dict)
+ and filter_.get("isExtra")
+ and filter_ not in secondary_filters
+ )
+ layer["adhoc_filters"] = secondary_filters
+ split_adhoc_filters_into_base_filters(layer, engine)
+ layer_metrics = _timeseries_base_metrics(layer)
+ layer_groupby = _as_list(layer.get("groupby"))
+ columns = [*(_as_list(x_axis) if x_axis else []), *layer_groupby]
+ query = build_single_query_dict(
+ layer,
+ _dedupe_query_fields(columns, _column_label),
+ layer_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=layer.get("orderby"),
+ )
+ query["series_columns"] = layer_groupby
+ if not x_axis:
+ query["is_timeseries"] = True
+ comparison = _time_comparison(layer, layer_metrics)
+ query["time_offsets"] = (
+ _as_list(layer.get("time_compare")) if comparison else []
+ )
+ query["post_processing"] = _timeseries_post_processing(
+ layer,
+ query,
+ operator_metrics=layer_metrics,
+ )
+ _normalize_orderby(query)
+ queries.append(query)
+ return queries
+
+
+def build_timeseries_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render ECharts Timeseries buildQuery and its post-processing."""
+ x_axis = _normalized_x_axis_query_field(form_data)
+ timeseries_metrics = _timeseries_base_metrics(form_data)
+ extra_metrics = _timeseries_extra_metrics(form_data)
+ timeseries_groupby = _as_list(form_data.get("groupby"))
+ columns = [*(_as_list(x_axis) if x_axis else []), *timeseries_groupby]
+ query = build_single_query_dict(
+ form_data,
+ _dedupe_query_fields(columns, _column_label),
+ [*timeseries_metrics, *extra_metrics],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ query["series_columns"] = timeseries_groupby
+ if not x_axis:
+ query["is_timeseries"] = True
+ comparison = _time_comparison(form_data, timeseries_metrics)
+ query["time_offsets"] = (
+ _as_list(form_data.get("time_compare")) if comparison else []
+ )
+ query["time_compare_full_range"] = bool(
+ query["time_offsets"] and form_data.get("time_compare_full_range")
+ )
+ query["post_processing"] = _timeseries_post_processing(
Review Comment:
Fixed in 4f9f6c1a814d57210b46fbfd479ec9f108f83528. The XY plugin supplies a shared saved/unsaved preview adapter that folds native wide pivot columns into separate metric/category series with color encoding. Regression tests execute the real query post-processing and verify both EU and US series for line and bar previews.
##########
superset/mcp_service/chart/compile.py:
##########
@@ -209,6 +240,22 @@ def _adhoc_filter_column_valid(
WHERE filters must reference a physical column; HAVING filters may also
reference a saved metric because Superset resolves metric names there.
"""
+ column_names = [item["name"] for item in dataset_context.available_columns]
+ metric_names = [metric["name"] for metric in dataset_context.available_metrics]
+ if column in column_names or (clause == "HAVING" and column in metric_names):
+ return True
+ try:
+ if resolve_dataset_column(column, dataset_context) is not None:
+ return True
+ except ValueError:
Review Comment:
Fixed in 4f9f6c1a814d57210b46fbfd479ec9f108f83528. Ambiguous physical references are re-raised before the generic ValueError handler, while HAVING retains its unambiguous saved-metric fallback. The unreachable code is removed. Restored WHERE/HAVING regressions verify AMBIGUOUS_DATASET_REFERENCE and both Score/SCORE candidates.
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,2010 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+_EXCEL_MIN_DATE = date(1900, 1, 1)
+# Leave room for XLSX readers' millisecond rounding at the final serial day.
+_EXCEL_MAX_TIME = time(23, 59, 59, 999000)
+_EXCEL_MAX_DATETIME = datetime.combine(date.max, _EXCEL_MAX_TIME)
+# Serial day 2958465 is 9999-12-31 in Excel's 1900 date system.
+_EXCEL_MAX_DURATION = timedelta(
+ days=2958465, hours=23, minutes=59, seconds=59, milliseconds=999
+)
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
return None
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget, *, row_shaped: bool = False
+) -> str | None:
+ """Bound metadata, charging row-shaped indexes to the row-data work budget."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if row_shaped:
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "metadata contains too many total values"
+ else:
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list or type(item) is tuple:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = len(item)
+ # Chart Data emits one indexname per row. Only that outer array
+ # receives the row limit; containers within an index stay bounded.
+ max_items = (
+ MAX_QUERY_RESULT_ROWS
+ if row_shaped and depth == 0
+ else _MAX_ROW_CONTAINER_ITEMS
+ )
+ if width > max_items:
+ return "metadata contains an oversized array"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(width, mapping=False),
+ metadata=True,
+ ):
+ return reason
+ stack.extend((item[index], depth + 1, False) for index in range(width))
+ continue
+
+ normalized, reason = _normalize_trusted_scalar(
+ item, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ )
+ if reason is not None:
+ return f"metadata {reason}"
+ if reason := _charge_json_bytes(
+ budget,
+ _normalized_scalar_json_size(
+ normalized, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ ),
+ metadata=True,
+ ):
+ return reason
+ return None
+
+
+def _metadata_failure( # noqa: C901
+ payload: dict[str, Any], label: str
+) -> ChartError | None:
+ """Validate bounded cache and row-count metadata before any consumer."""
+ for key in ("rowcount", "total_rows"):
+ if key in payload and dict.__getitem__(payload, key) is not None:
+ try:
+ bounded_result_row_count(dict.__getitem__(payload, key))
+ except ValueError as ex:
+ return _malformed_result(f"{label} {key} {ex}")
+
+ if "is_cached" in payload:
+ is_cached = dict.__getitem__(payload, "is_cached")
+ if is_cached is not None and type(is_cached) is not bool:
+ return _malformed_result(f"{label} is_cached must be an exact boolean")
+
+ if "cache_key" in payload:
+ cache_key = dict.__getitem__(payload, "cache_key")
+ if cache_key is not None and (
+ type(cache_key) is not str
+ or _bounded_utf8_length(cache_key, _MAX_CACHE_STRING_BYTES) is None
+ ):
+ return _malformed_result(
+ f"{label} cache_key must be a bounded exact string"
+ )
+
+ # ChartData's production schema emits ``cached_dttm``. ``cache_dttm`` was
+ # used by earlier MCP payloads and remains a bounded compatibility alias.
+ # Validate both before cache utilities parse or compare either value.
+ for key in ("cached_dttm", "cache_dttm"):
+ if key not in payload:
+ continue
+ cache_dttm = dict.__getitem__(payload, key)
+ if cache_dttm is None:
+ continue
+ if type(cache_dttm) is str:
+ if _bounded_utf8_length(cache_dttm, _MAX_CACHE_STRING_BYTES) is not None:
+ continue
+ return _malformed_result(f"{label} {key} must be a bounded exact string")
+ if type(cache_dttm) is datetime:
+ tzinfo = cache_dttm.tzinfo
+ if tzinfo is None or any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ continue
+ return _malformed_result(f"{label} {key} has an unsupported timezone")
+ return _malformed_result(
+ f"{label} {key} must be a bounded exact string or datetime"
+ )
+ return None
+
+
+def _excel_temporal_cell(value: Any) -> Any | None:
+ """Return a naive builtin temporal value an XLSX writer stores natively.
+
+ Called only after ``_normalize_trusted_scalar`` accepted ``value``. XLSX has
+ no timezone-aware date cells, so aware values keep their ISO text, as do
+ NumPy durations whose unit may be ambiguous. Values outside the 1900 date
+ system's range, including negative durations, also keep their ISO text.
+ """
+ value_type = type(value)
+ if value_type is date:
+ return value if value >= _EXCEL_MIN_DATE else None
+ if value_type is timedelta:
+ return value if timedelta(0) <= value <= _EXCEL_MAX_DURATION else None
+ if value_type is datetime:
+ return (
+ value
+ if value.tzinfo is None
+ and value.date() >= _EXCEL_MIN_DATE
+ and value <= _EXCEL_MAX_DATETIME
+ else None
+ )
+ if value_type is time:
+ return value if value.tzinfo is None and value <= _EXCEL_MAX_TIME else None
+ if value_type is np.datetime64:
+ value = pd.Timestamp(value)
+ value_type = pd.Timestamp
+ if value_type is pd.Timestamp:
+ if value.tzinfo is not None or not 1900 <= value.year <= 9999:
+ return None
+ # Excel stores millisecond precision; drop nanoseconds explicitly
+ # instead of through pandas' lossy-conversion warning.
+ return _excel_temporal_cell(
+ datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ )
+ if value_type is pd.Timedelta:
+ if not 0 <= value.days <= _EXCEL_MAX_DURATION.days:
+ return None
+ return _excel_temporal_cell(
+ timedelta(
+ days=value.days, seconds=value.seconds, microseconds=value.microseconds
+ )
+ )
+ return None
+
+
+def _normalize_row_value( # noqa: C901
+ value: Any,
+ budget: _ResultBudget,
+ *,
+ temporal_json_numbers: bool = False,
+ preserve_nonfinite_floats: bool = False,
+ preserve_excel_temporals: bool = False,
+) -> str | None:
+ """Normalize trusted scalars and return a bounded serialization failure.
+
+ ChartDataCommand materializes a real DataFrame with ``to_dict(records)``, so
+ exact pandas and NumPy scalars can remain in an otherwise valid result. They
+ are converted in place to their canonical JSON-facing builtin values here.
+ Exact containers are inspected through builtin operations only; arbitrary
+ subclasses and scalar hooks remain rejected. ``preserve_excel_temporals``
+ keeps naive row-level date, time, datetime, and duration cells as builtin
+ temporal objects for XLSX exports; size budgets still use their JSON text.
+ """
+ stack: list[
+ tuple[
+ Any,
+ list[Any] | dict[str, Any] | None,
+ int | str | None,
+ int,
+ bool,
+ ]
+ ] = [(value, None, None, 0, False)]
+ active_containers: set[int] = set()
+
+ while stack:
+ item, parent, slot, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "exceeds the nesting depth limit"
+
+ if type(item) is np.ndarray:
+ # Arrow list cells are exact arrays. Expand one bounded dimension at
+ # a time, never calling tolist() before shape/work checks or trusting
+ # ndarray subclasses. Object-array children use the same scalar and
+ # cycle guards as builtin containers.
+ identity = id(item)
+ if identity in active_containers:
+ return "contains cyclic containers"
+ if depth + item.ndim > _MAX_ROW_CONTAINER_DEPTH:
+ return "exceeds the nesting depth limit"
+ if any(width > _MAX_ROW_CONTAINER_ITEMS for width in item.shape):
+ return "contains an oversized array"
+ if item.size > MAX_QUERY_RESULT_VALUES - budget.values:
+ return "exceeds the total value limit"
+ active_containers.add(identity)
+ stack.append((item, None, None, depth, True))
+ scalar_array = item.ndim == 0
+ item = (
+ np.ndarray.__getitem__(item, ())
+ if scalar_array
+ else [
+ np.ndarray.__getitem__(item, index)
+ for index in range(item.shape[0])
+ ]
+ )
+ if type(parent) is list:
+ assert type(slot) is int
+ list.__setitem__(parent, slot, item)
+ elif type(parent) is dict:
+ assert type(slot) is str
+ dict.__setitem__(parent, slot, item)
+ if scalar_array:
+ stack.append((item, parent, slot, depth + 1, False))
+ continue
+
+ if type(item) is list or type(item) is tuple:
+ identity = id(item)
+ if identity in active_containers:
+ return "contains cyclic containers"
+ width = len(item)
+ if width > _MAX_ROW_CONTAINER_ITEMS:
+ return "contains an oversized array"
+ active_containers.add(identity)
+ stack.append((item, None, None, depth, True))
+ if type(item) is tuple:
+ item = list(item)
+ if type(parent) is list:
+ assert type(slot) is int
+ list.__setitem__(parent, slot, item)
+ elif type(parent) is dict:
+ assert type(slot) is str
+ dict.__setitem__(parent, slot, item)
+ if reason := _add_result_value_to_budget(item, budget):
+ return reason
+ if reason := _charge_json_bytes(
+ budget, _container_json_syntax_size(width, mapping=False)
+ ):
+ return reason
+ stack.extend(
+ (list.__getitem__(item, index), item, index, depth + 1, False)
+ for index in range(width)
+ )
+ continue
+
+ if type(item) is dict:
+ if reason := _add_result_value_to_budget(item, budget):
+ return reason
+ identity = id(item)
+ if identity in active_containers:
+ return "contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, None, None, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget, _container_json_syntax_size(item_count, mapping=True)
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size):
+ return reason
+ stack.append((child, item, key, depth + 1, False))
+ continue
+
+ normalized: Any
+ generic_scalar = False
+ if (
+ preserve_nonfinite_floats
+ and type(item) is float
+ and not math.isfinite(item)
+ ):
+ normalized, reason = item, None
+ elif temporal_json_numbers and _is_chart_data_temporal_scalar(item):
+ normalized, reason = _chart_data_temporal_number(item)
+ elif temporal_json_numbers and _is_chart_data_duration_scalar(item):
+ normalized, reason = _chart_data_duration_text(item)
+ else:
+ normalized, reason = _normalize_trusted_scalar(item)
+ generic_scalar = True
+ if reason is not None:
+ return reason
+ if reason := _add_result_value_to_budget(normalized, budget):
+ return reason
+ replacement = normalized
+ # Plugins with Chart Data temporal wire semantics (numbers/duration
+ # text) keep that projection in every export format.
+ if (
+ preserve_excel_temporals
Review Comment:
Fixed in 4f9f6c1a814d57210b46fbfd479ec9f108f83528. CSV preserves established trusted row-cell temporal spellings on saved and cached chart-data paths, without changing JSON or Bullet temporal wire semantics. Regressions cover datetime, duration, pandas/NumPy cells, and timezone-aware values; hook-free validation and size budgets remain enforced.
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1349,6 +1359,111 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Inherited native SQL dimensions remain query expressions when updating metrics
+or `order_by`; reference their output labels to sort by them. Caller-supplied
+`dimensions` must still be physical columns, not SQL expressions.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling. Bullet metric strings use
+JavaScript numeric spellings: underscore separators and non-ASCII digits are
+rejected rather than interpreted as numbers. Bounded array-valued dimensions
+retain their raw values in data reads and exports; preview category labels use
+JavaScript string conversion (for example, `[1, 2]` displays as `1,2`). Native
+SQL metrics without a label use their SQL expression as the output label.
+
+Presentation-only updates preserve saved predicates, including legacy top-level
+`where`, `having`, and `filters`, and normalize the native `order_by_cols` alias.
+When both `orderby` and `order_by_cols` are saved, their sort entries are
+concatenated in form-data key order, including when `orderby` is empty.
+Explicit `filters: []` and `order_by: []` clear those inherited controls.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+Labels that become empty after sanitization retain their slots so later labels
+remain aligned with their corresponding values.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates. Omitted label controls preserve the saved state only when their
+corresponding value controls (`ranges`, `markers`, or `marker_lines`) are also
+omitted. Replacing a value control without its labels clears the saved labels;
+resend the labels to retain annotations with the replacement values.
+
+Saved native `ranges`, `markers`, and `marker_lines` controls ignore empty,
+non-numeric, and NaN tokens, matching Explore. If no numeric range remains, the
+preview uses the default band up to 110% of the largest measure. Unrelated
+updates preserve these saved controls. Newly authored typed lists must contain
+finite numbers; infinite native values and oversized controls remain errors.
+
+On same-dataset chart updates, `filters: []` clears both user filters and the
+generated dashboard-time binding. `temporal_column: null` clears only that
+generated binding, preserving user filters. Omitting those controls preserves
+the saved binding. Bullet Vega previews keep separate indexed rows even when
+dimension display labels are identical, and support both `SMART_NUMBER` and
+`SMART_NUMBER_SIGNED` number formats. Bullet numeric format precision is limited
+to 20 digits before formatting, including on data reads.
+
+Native Bullet temporal filters retain the active filter's subject and range as a
+pair; `No filter` placeholders do not supply the subject of another active range.
+Typed configs support one such pair. Multiple active native temporal filters, or
+conflicts with explicit `temporal_column`/`time_range`, are rejected rather than
+silently dropping or moving a predicate.
+
+### Query result limits
+
+MCP chart tools accept up to 50,000 rows per query. The row-shaped `indexnames`
+array emitted by Chart Data uses the same 50,000-entry limit, rather than the
+4,096-entry limit for other metadata arrays. Index entries count toward the
+shared row-data work budget and retain the 1 MiB aggregate metadata byte limit.
+Multi-dimension pivot index tuples are serialized as JSON arrays. Nested
+containers within an index entry retain the standard container limits.
+
+The shared query-result validator also enforces **non-configurable hard limits**:
+
+- **64 KiB (65,536 UTF-8 bytes) per text cell/string value**. A single
Review Comment:
Fixed in 4f9f6c1a814d57210b46fbfd479ec9f108f83528. The MCP documentation and UPDATING.md distinguish the 64 KiB row-cell string cap from the separate aggregate metadata budget, including SQL text. They also clarify that the 20-digit Bullet format-precision limit applies to previews, not raw data reads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4224693520
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,112 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ prefixes = {
+ spelling
+ for label in metric_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(prefix + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(prefix + "__")
+ for prefix in prefixes
+ )
+ ]
+ if not fields and len(metric_labels) == 1 and form_data.get("truncate_metric"):
+ # A single truncated metric drops its label from the pivoted column
+ # names, so every non-x-axis column is one category series.
+ fields = [field for field in data[0] if field != x_axis]
+ if not fields:
+ return None
+ sample = data[0][x_axis]
+ x_type = (
+ "temporal"
+ if isinstance(sample, str) and any(char in sample for char in "-/: ")
+ else "quantitative"
+ if isinstance(sample, (int, float))
+ else "nominal"
+ )
+ return VegaLitePreview(
+ specification={
+ "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
+ "data": {"values": data},
+ "transform": [
+ {
+ "fold": [
+ "".join(
+ "\\" + char if char in ".[]\\" else char for char in field
+ )
+ for field in fields
+ ],
+ "as": ["__mcp_xy_series", "__mcp_xy_value"],
+ }
+ ],
+ "mark": mark,
+ "encoding": {
+ "x": {"field": x_axis, "type": x_type, "title": x_axis},
Review Comment:
Confirmed. Fixed in 42e861005ca93f4d00cb60f87478eeff98fd32eb: the x encoding and the x tooltip field use the same `.`/`[`/`]`/`\` escaping as the folded series (one `vega_field` helper), while titles keep the raw name. New `test_xy_preview_escapes_dotted_x_axis_field` uses a real `orders.date` x-axis result and asserts `orders\.date` in both the encoding and tooltip; it fails without the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5963211674
@rusackas quick clarification on the Gantt `allows_empty_result` note: the flag is on master now (it came in with #44746), and this branch no longer changes gantt.py's `allows_empty_result` relative to master, so there is no longer a Gantt behavior difference in this PR. The saved-chart plugin-contract scope for update_chart / update_chart_preview is in place and covered by the disabled-Treemap tests. Head 88d884fbe0 has all checks green, no unresolved threads, and a clean merge with master. Re-requested your review earlier; happy to adjust anything else.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4246334772
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1537,6 +1537,71 @@ including an explicit clear.
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
Review Comment:
Fixed in cad4fcf56f4a30a400b455f809e3b3db48f685a1. Moved the entire unchanged “Bullet chart compatibility” H2 section immediately before “Disabling chart type plugins”. The four existing H3 sections remain under “Next Steps”, matching master; no other content was reordered. Pre-commit passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4240083048
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -3489,7 +3520,643 @@ def _as_column_list(value: Any) -> list[Any]:
return [value]
-def analyze_chart_capabilities(viz_type: str | None, config: Any) -> ChartCapabilities:
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ # A missing saved viz_type is a boundary too, matching merge_chart_form_data.
+ if existing_form_data.get("viz_type") != new_form_data.get("viz_type"):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+ if (
+ existing_binding is None
+ and existing_subject is None
+ and range_explicit
+ and not subject_authoritative
+ and "filters" not in explicit_fields
+ ):
+ # A range-only update on an unmarked chart adopts its saved temporal
+ # filter even when Explore bound it to a column other than the mapper's
+ # proposed subject. Zero or several native ranges stay ambiguous, so the
+ # proposed binding is added instead of guessing which one to rewrite.
+ unmarked_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("expressionType") == "SIMPLE"
+ and filter_.get("clause") == "WHERE"
+ and isinstance(filter_.get("subject"), str)
+ ]
+ if len(unmarked_matches) == 1:
+ existing_binding = unmarked_matches[0]
+ existing_subject = existing_binding["subject"]
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet controls without reinterpreting inherited query roles.
+
+ Saved Explore state may contain SQL dimensions and SQL WHERE/SIMPLE HAVING
+ filters beyond the typed authoring surface. Omitted roles are validated by
+ the native query contract and compilation, not as newly authored physical
+ columns or SIMPLE WHERE filters. Only this validation copy excludes them
+ and native query metadata; compiled and persisted form data stays intact.
+ Explicit replacements, including ``[]``, retain strict typed validation.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ for native_query_key in ("url_params", "extra_form_data", "extra_filters"):
+ validation_data.pop(native_query_key, None)
+ if not validation_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT):
+ # A null marker records an explicit subject clear with no binding to
+ # validate; it is not an authored physical column.
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ if update_config is None or isinstance(update_config, BulletChartConfig):
+ if update_config is None or update_config.dimensions is None:
+ validation_data.pop("groupby", None)
+ if update_config is None or "filters" not in update_config.model_fields_set:
+ validation_data.pop("adhoc_filters", None)
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ if update_config is None or "order_by" not in update_config.model_fields_set:
+ # An inherited sort may rank by an independent saved metric; the
+ # native query contract validates it, not the authoring schema.
+ for order_key in ("orderby", "order_by_cols"):
+ validation_data.pop(order_key, None)
+ return BulletChartConfig.model_validate(validation_data)
Review Comment:
Valid, fixed in 50b3811761b1f94801a4986242cce48f3a8ea337. The validation copy now drops the envelope/presentation/time keys from `FORM_DATA_UPDATE_PRESERVE_KEYS` that `BulletChartConfig` does not model (`show_legend`/`time_range` stay validated). Regression: `test_bullet_update_from_other_viz_ignores_preserved_unmodeled_keys` converts a saved Pie with `color_scheme`, `number_format`, `legendOrientation`, etc. through both the save and preview-first paths. It also checks that a typo like `colour_scheme` is still rejected.
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -902,6 +912,9 @@ async def _get_chart_data( # noqa: C901
if result_error := validate_query_result_envelope(
result,
none_as_empty=null_data_is_empty(effective_form_data.get("viz_type")),
+ temporal_json_numbers=temporal_json_numbers(
Review Comment:
Valid, fixed in 50b3811761b1f94801a4986242cce48f3a8ea337. `_excel_temporal_value` now restores int/float epoch-millisecond cells in temporal columns as naive UTC datetimes. It keeps the same Excel range guards, and non-finite or out-of-range values are left unchanged. Regression: `test_saved_bullet_excel_restores_epoch_temporal_cells_as_dates` (openpyxl and xlsxwriter) groups a saved Bullet by a temporal `order_date`. Before the fix the cell held `1704153600000`; it is now `datetime(2024, 1, 2)`.
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,589 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from copy import deepcopy
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ bullet_groupby_list,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ resolve_bullet_order_target,
+ TablePreview,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Validate Bullet result rows without requiring a backend preview formatter."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data), validate_format=False)
+ except BulletOutputError as ex:
+ return ChartError(error=bounded_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+def _unsupported_table_preview() -> ChartError:
+ return ChartError(
+ error=(
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ ),
+ error_type="UnsupportedFormat",
+ )
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # A null ``data`` array is a malformed producer result, not an empty chart.
+ null_data_is_empty = False
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ validates_native_references = True
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ # A merged validation copy omits native dimensions; its native sorts
+ # belong to query-contract validation, not physical-column resolution.
+ orders = (
+ config_dict.get("order_by") or []
+ if config.dimensions is not None or config._inherited_groupby is not None
+ else []
+ )
+ for order in orders:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.order_dimensions, config.metric
+ )
+ if role == "dimension" and index is not None:
+ if config.dimensions is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized._inherited_groupby = deepcopy(config._inherited_groupby)
+ normalized.validate_roles_and_outputs()
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def build_query_dicts(
+ self,
+ form_data: dict[str, Any],
+ *,
+ viz_type: str,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+ ) -> list[dict[str, Any]] | None:
+ """Mirror ``Bullet/buildQuery``: one metric over the groupby hierarchy.
+
+ The frontend keeps only explicitly saved ordering; it never applies
+ the legacy top-N metric ordering of unadapted visualizations.
+ """
+ from superset.common.form_data_query_context import (
+ build_query_objects_from_form_data,
+ )
+
+ queries = build_query_objects_from_form_data(
+ form_data,
+ viz_type=viz_type,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ filters_prepared=True,
+ )
+ query = queries[0]
+ metric = form_data.get("metric")
+ query["columns"] = bullet_groupby_list(form_data.get("groupby"))
+ query["metrics"] = [metric] if metric else []
+ if not form_data.get("orderby") and not form_data.get("order_by_cols"):
+ query["orderby"] = []
+ return queries
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import first_query_data
+
+ rows, failure = first_query_data(
+ result, none_as_empty=False, temporal_json_numbers=True
+ )
+ if failure is not None:
+ return failure
+ assert rows is not None
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose validated rows independently of backend preview formatter support."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ model = resolve_bullet_render_model(
+ data, dict(form_data), validate_format=False
+ )
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+ if not data:
+ # The strict model's zero-valued ungrouped row is a render-only
+ # frontend fallback, not source query data, so an empty query
+ # exposes no rows to get-data and exports.
+ return [], None
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the dimensions. Its float measure is
+ # render-only: exported rows keep the validated source metric so exact
+ # BIGINT/Decimal values are not rounded to binary64.
+ # An ungrouped model keeps only the rendered first row; every later
+ # source row was validated by the same resolver and stays exported.
+ model_rows = list(model.rows)
+ if not model.dimensions:
+ try:
+ for source in data[len(model_rows) :]:
+ model_rows.extend(
+ resolve_bullet_render_model(
+ [source], dict(form_data), validate_format=False
+ ).rows
+ )
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+ rows: list[Any] = []
+ for source, row in zip(data, model_rows, strict=True):
+ exposed = dict(row)
+ if model.metric_field in source:
+ exposed[model.metric_field] = _safe_enum_backing(
+ source[model.metric_field]
+ )
+ rows.append(exposed)
+ return rows, None
+
+ def ascii_preview(
+ self,
+ data: list[Any],
+ form_data: dict[str, Any],
+ width: int,
+ height: int = 20,
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
Review Comment:
Valid, fixed in 50b3811761b1f94801a4986242cce48f3a8ea337. The clipping is factored into `preview_utils.fit_ascii_frame`. The saved-chart `ASCIIPreviewStrategy` now applies it to plugin output, so the content matches the width/height it reports, the same way the unsaved path does. Regression: `test_saved_bullet_ascii_preview_fits_requested_frame` calls `get_chart_preview` with `ascii_width=40, ascii_height=5`. It returned 13 lines before the fix and now returns at most 5 lines of at most 40 columns.
##########
superset/mcp_service/chart/plugin.py:
##########
@@ -315,6 +318,24 @@ def schema_error_hint(self) -> ChartGenerationError | None:
invalid_result_message: ClassVar[str]
invalid_result_suggestions: ClassVar[tuple[str, ...]]
+ #: Whether chart-data consumers convert temporal and duration result
+ #: values to the JSON numbers and text the frontend ``transformProps``
+ #: receives, instead of the generic scalar serialization.
+ temporal_json_numbers: ClassVar[bool]
+
+ #: Whether raw get_chart_data rows and CSV/XLSX exports keep exact
+ #: non-finite floats (JSON responses still sanitize them to null).
+ preserve_nonfinite_floats: ClassVar[bool]
Review Comment:
Valid, fixed in 50b3811761b1f94801a4986242cce48f3a8ea337. The flag was left over from a pre-merge Gauge path. No plugin sets it, and master's result normalization has no consumer for it, so I removed it from the protocol and the base class. Regression: `test_every_declared_plugin_class_flag_is_read` fails for any `ClassVar` declared on `BaseChartPlugin` that nothing under `superset/mcp_service` reads. The two upstream flags that are unread on master are allowlisted explicitly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4129946199
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -581,6 +614,12 @@ async def execute_chart_data( # noqa: C901
# The query_context contains all the information needed to reproduce
# the chart's data exactly as shown in the visualization
query_context_json = None
+ form_data: dict[str, Any] = {}
+ if chart_params:
+ parsed_form_data = utils_json.loads(chart_params)
Review Comment:
Fixed in 70611d12b4f566cd690093318d5159cf28caa57c: removed the duplicate unguarded parse and reused the guarded saved-params path, so malformed legacy params no longer block a valid saved query context. The regression test covers malformed JSON.
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -792,11 +833,21 @@ async def execute_chart_data( # noqa: C901
command.validate()
result = command.run()
- if form_data.get("viz_type") == "treemap_v2":
+ if _normalizes_data_results(form_data):
result = normalize_chart_query_result(result, form_data)
if isinstance(result, ChartError):
return result
- if query_failure := query_result_failure(result):
+ data_plugin = _data_plugin(chart_viz_type)
Review Comment:
Fixed in 70611d12b4f566cd690093318d5159cf28caa57c: resolved the data plugin from the effective form data so cached type changes use the queried plugin’s empty-result and row-shape contract. Added coverage for an empty Bullet preview overriding a saved table chart.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2656,6 +2740,426 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ data.setdefault("temporal_column", subject)
+ if isinstance(comparator, str) and comparator.casefold() != "no filter":
+ data.setdefault("time_range", comparator)
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
+ {"name": item} if isinstance(item, str) else item
+ for item in data[key]
+ ]
+ for key in ("order_by", "orderby", "order_by_cols"):
+ if key in data:
+ data[key] = cls._adapt_native_order_by(data[key])
+ cls._adapt_native_filters(data)
+ return data
+
+ @field_validator("ranges", "markers", "marker_lines", mode="before")
+ @classmethod
+ def tokenize_native_numeric_lists(cls, value: Any) -> Any:
+ """Parse numeric controls without creating values for empty tokens."""
+ if value is None:
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",") if token.strip()]
+ return value
+
+ @field_validator(
+ "range_labels", "marker_labels", "marker_line_labels", mode="before"
+ )
+ @classmethod
+ def tokenize_native_label_lists(cls, value: Any) -> Any:
+ """Parse label controls while preserving positional empty tokens."""
+ if value is None or value == "":
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",")]
+ return value
+
+ @field_validator("ranges", "markers", "marker_lines")
+ @classmethod
+ def reject_non_finite_values(cls, values: List[float]) -> List[float]:
+ if any(not math.isfinite(value) for value in values):
+ raise ValueError("Bullet thresholds and markers must be finite numbers")
+ return values
+
+ @field_validator("range_labels", "marker_labels", "marker_line_labels")
+ @classmethod
+ def validate_presentation_labels(cls, labels: List[str]) -> List[str]:
+ result: list[str] = []
+ for label in labels:
+ if "," in label:
+ raise ValueError(
+ "Bullet labels cannot contain commas because the frontend "
+ "comma-separated controls have no escaping"
+ )
+ if label == "":
+ result.append("")
+ continue
+ sanitized = sanitize_user_input(
+ label, "Bullet label", max_length=200, allow_empty=True
+ )
+ if sanitized is not None:
+ result.append(sanitized)
+ return result
+
+ @field_validator("time_range")
+ @classmethod
+ def sanitize_time_range(cls, value: str | None) -> str | None:
+ return sanitize_user_input(
+ value, "Time range", max_length=1000, allow_empty=True
+ )
+
+ @model_validator(mode="after")
+ def validate_roles_and_outputs(self) -> "BulletChartConfig": # noqa: C901
+ dimensions = self.dimensions or []
+ seen_names: set[str] = set()
+ for index, dimension in enumerate(dimensions):
+ _reject_sql_expression_on_dimension(dimension, f"dimensions[{index}]")
+ if dimension.saved_metric or dimension.aggregate:
+ raise ValueError(
+ f"dimensions[{index}] must be a physical dimension, not a metric"
+ )
+ name = dimension.name or ""
+ if name in seen_names:
+ raise ValueError(f"Duplicate Bullet dimension: {name!r}")
+ seen_names.add(name)
+
+ # ``map_bullet_config`` deliberately emits physical groupby names. The
+ # frontend therefore reads dimension results under those names even when
+ # a friendly ``ColumnRef.label`` was supplied. Only the metric is emitted
+ # with an output alias. Keep validation aligned with those actual result
+ # fields instead of treating dimension display labels as SQL aliases.
+ if (metric_output := _bullet_metric_output_label(self.metric)) in seen_names:
+ raise ValueError(
+ f"Bullet metric output label {metric_output!r} conflicts with a "
+ "dimension (its physical output name); provide a unique metric label"
+ )
+
+ resolved_order: list[tuple[str, int | None]] = []
+ for item in self.order_by:
Review Comment:
Fixed in 70611d12b4f566cd690093318d5159cf28caa57c: deferred Bullet sort cross-checks until the saved hierarchy is resolved before mapping in both update tools. Tests cover omitted/null dimensions, unknown targets, rebinds, and physical-dimension precedence over metric aliases.
##########
superset/mcp_service/chart/plugins/gauge.py:
##########
@@ -44,6 +49,12 @@ class GaugeChartPlugin(BaseChartPlugin):
native_viz_types: ClassVar[Mapping[str, str]] = {
"gauge_chart": "Gauge Chart",
}
+ requires_compile_check = True
+ strict_dataset_rebind = True
Review Comment:
Fixed in 70611d12b4f566cd690093318d5159cf28caa57c: set Gauge’s owns_update_merge flag so generic preservation cannot restore intentionally cleared bindings. Regression tests cover null time_range and granularity_sqla in save, preview-first, and cached-preview updates.
##########
superset/mcp_service/chart/tool/update_chart.py:
##########
@@ -897,7 +975,6 @@ async def update_chart( # noqa: C901
if validation_config is not None and effective_norm_dataset_id is not None:
from superset.mcp_service.chart.validation.dataset_validator import (
DatasetValidator,
Review Comment:
Fixed in 70611d12b4f566cd690093318d5159cf28caa57c: removed the unused NORMALIZATION_EXCEPTIONS constant and its stale shared-handling comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4143956864
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1839 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Integer/Decimal bounds also prevent
+# later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget
+) -> str | None:
+ """Bound exact-container query metadata independently of row data."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = list.__len__(item)
+ if width > _MAX_ROW_CONTAINER_ITEMS:
Review Comment:
Confirmed: Chart Data emits `indexnames = list(df.index)`, and a 5,000-row result failed the generic metadata array cap.
Fixed in 7d093a8137a0a926861a828f3947e572866af537. The query-level outer `indexnames` array uses `MAX_QUERY_RESULT_ROWS` (50,000). Index entries share the row-data value/work budget, avoiding the separate 32,768 metadata-item ceiling, while retaining the 1 MiB metadata byte budget. Other metadata arrays and nested index containers keep their existing bounds.
Regression tests cover 5,000 and 50,000 rows, two full-size query legs, rejection above the row limit, unchanged generic/nested array caps, and value/work/metadata-byte accounting. The new acceptance tests failed before the fix. Final MCP chart suite: 3,329 passed, 3 skipped. Pre-commit passed on touched files and all branch-changed files.
Also merged upstream/master in a240236b8de55b1e27d72955fe5d3623c54db723, preserving both conflicting UPDATING.md entries and the plugin lifecycle/Bullet behavior. The post-merge chart suite passed before the index fix as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5857563012
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5920295009
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5987609699
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5995596118
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5989455411
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5606255246
Finite maintainer pass completed at `640f1210365d67d4cee3f4ea1c700afa5c022a77`.
- Fetched all live reviews, issue comments, inline review comments, and all 12 review threads. No new actionable feedback was posted; all 12 threads remain resolved, so no thread state was changed.
- Normally merged upstream master through `70346a51` and pushed to Amin's fork. No rebase, force-push, or history rewrite.
- Verified the branch remains MERGEABLE and non-draft; BLOCKED reflects required review/check state.
- Post-merge Bullet/Gauge/shared MCP regression run: **659 passed, 1 skipped**. `git diff --check origin/master...HEAD` passed and the worktree is clean.
- Updated the PR body with the exact head and removed the stale private tracking reference.
Exact-head CI is not fully green yet. All completed required checks except Playwright are passing; MySQL remains in progress. The Playwright matrix failed during environment setup because apt package indexes could not be downloaded (exit 100), before tests ran. I retried the failed jobs twice; both retries hit the same external package-index failure. The required aggregator therefore remains red despite no test failure: https://github.com/apache/superset/actions/runs/34382264261
This is the end of the requested finite pass; no merge or draft-state change was performed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5779249339
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4203416591
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,847 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
Review Comment:
`BulletChartConfig.row_limit` defaults to 10,000, but this only emits it when it was explicitly supplied, including when creating a new chart. A new Bullet saved without `row_limit` therefore carries none, so `get_chart_data` falls back to the deployment `ROW_LIMIT` (50,000 by default), and a grouped Bullet with 20,000 categories returns all 20,000 rows rather than the advertised 10,000 maximum. Could creation apply the schema default while update-time omission still preserves the saved value?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,847 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if isinstance(existing_viz_type, str) and existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+def _currency_form_value(value: CurrencyFormat | None) -> dict[str, str] | None:
+ """Return the native value for an explicitly supplied currency control."""
+ return value.to_form_data() if value is not None else None
+
+
+def _column_names(value: Sequence[ColumnRef] | None) -> list[str | None] | None:
+ """Return a native column-name list while retaining an explicit null."""
+ return [column.name for column in value] if value is not None else None
+
+
+def _table_sort_value(value: Sequence[str | SortByConfig] | None) -> list[str] | None:
+ """Return the native Table sort control for an explicit typed value."""
+ if value is None:
+ return None
+ return [
+ json.dumps(
+ [entry.column, entry.ascending]
+ if isinstance(entry, SortByConfig)
+ else [entry, False]
+ )
+ for entry in value
+ ]
+
+
+def _table_column_config_value(value: Any) -> dict[str, Any] | None:
+ """Return Table column config without losing an explicit null or empty map."""
+ if value is None:
+ return None
+ return {
+ label: column.model_dump(by_alias=True, exclude_unset=True)
+ for label, column in value.items()
+ }
+
+
+# Mappers intentionally omit optional controls so fresh charts use the frontend
+# defaults. During a same-viz update, however, an explicitly supplied false,
+# null, or empty value must block preservation of the saved native key. Keep the
+# typed-to-native relationship declarative so every update path shares it.
+_FormValueConverter = Callable[[Any], Any]
+_FormControlMap = dict[str, tuple[str, _FormValueConverter]]
+
+_COMMON_EXPLICIT_FORM_CONTROLS: _FormControlMap = {
+ "color_scheme": ("color_scheme", lambda value: value),
+ "currency_format": ("currency_format", _currency_form_value),
+ "show_value": ("show_value", lambda value: value),
+}
+
+_CHART_EXPLICIT_FORM_CONTROLS: dict[str, _FormControlMap] = {
+ "table": {
+ "sort_by": ("order_by_cols", _table_sort_value),
+ "column_config": ("column_config", _table_column_config_value),
+ },
+ "xy": {
+ "group_by": ("groupby", _column_names),
+ "series_limit": ("series_limit", lambda value: value),
+ "stacked": ("stack", lambda value: "Stack" if value else None),
+ "orientation": ("orientation", lambda value: value),
+ "legend_orientation": ("legendOrientation", lambda value: value),
+ "x_axis_time_format": ("x_axis_time_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "mixed_timeseries": {
+ "group_by": ("groupby", _column_names),
+ "group_by_secondary": ("groupby_b", _column_names),
+ "currency_format_secondary": (
+ "currency_format_secondary",
+ _currency_form_value,
+ ),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "waterfall": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "big_number": {
+ "subheader": ("subheader", lambda value: value),
+ "y_axis_format": ("y_axis_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "compare_lag": ("compare_lag", lambda value: value),
+ "time_format": ("time_format", lambda value: value),
+ "aggregation": ("aggregation", lambda value: value),
+ },
+ "handlebars": {
+ "style_template": ("styleTemplate", lambda value: value),
+ "columns": ("all_columns", _column_names),
+ "groupby": ("groupby", _column_names),
+ "metrics": ("metrics", _column_names),
+ },
+ "pivot_table": {
+ "date_format": ("date_format", lambda value: value),
+ },
+ "interactive_pivot": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "series_limit": ("series_limit", lambda value: value),
+ "date_format": ("date_format", lambda value: value),
+ "column_sort": ("colOrder", lambda value: value),
+ },
+}
+
+
+def _apply_explicit_form_controls( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply typed controls whose mapper omission represents a native clear."""
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ controls = {
+ **_COMMON_EXPLICIT_FORM_CONTROLS,
+ **_CHART_EXPLICIT_FORM_CONTROLS.get(config.chart_type, {}),
+ }
+ for field_name, (native_key, convert) in controls.items():
+ if field_name in explicit_fields:
+ converted = convert(getattr(config, field_name))
+ is_clear = (
+ converted is None
+ or converted is False
+ or converted == ""
+ or converted in ([], {})
+ )
+ if is_clear:
+ new_form_data[native_key] = converted
+
+ axis_controls = {
+ "xy": (
+ ("x_axis", "x_axis_title", "x_axis_format", None),
+ ("y_axis", "y_axis_title", "y_axis_format", "logAxis"),
+ ),
+ "mixed_timeseries": (
+ ("x_axis", "xAxisTitle", "x_axis_time_format", None),
+ ("y_axis", "yAxisTitle", "y_axis_format", "logAxis"),
+ (
+ "y_axis_secondary",
+ "yAxisTitleSecondary",
+ "y_axis_format_secondary",
+ "logAxisSecondary",
+ ),
+ ),
+ }
+ if config.chart_type in axis_controls:
+ for field_name, title_key, format_key, scale_key in axis_controls[
+ config.chart_type
+ ]:
+ if field_name not in explicit_fields:
+ continue
+ axis = getattr(config, field_name)
+ if axis is None:
+ new_form_data[title_key] = None
+ new_form_data[format_key] = None
+ if scale_key:
+ new_form_data[scale_key] = None
+ continue
+ axis_fields = set(axis.model_fields_set)
+ if "title" in axis_fields:
+ new_form_data[title_key] = axis.title
+ if "format" in axis_fields:
+ new_form_data[format_key] = axis.format
+ if scale_key and "scale" in axis_fields:
+ new_form_data[scale_key] = (
+ None if axis.scale is None else axis.scale == "log"
+ )
+
+ if config.chart_type == "xy" and "legend" in explicit_fields:
+ legend = config.legend
+ if legend is None:
+ new_form_data["show_legend"] = None
+ new_form_data["legendOrientation"] = None
+ else:
+ legend_fields = set(legend.model_fields_set)
+ if "show" in legend_fields:
+ new_form_data["show_legend"] = legend.show
+ if "position" in legend_fields:
+ new_form_data["legendOrientation"] = legend.position
+
+ if config.chart_type == "interactive_pivot":
+ if "temporal_column" in explicit_fields and config.temporal_column is None:
+ new_form_data["granularity_sqla"] = None
+ new_form_data["temporal_columns_lookup"] = None
+ if (
+ "series_limit_metric" in explicit_fields
+ and config.series_limit_metric is None
+ ):
+ new_form_data["series_limit_metric"] = None
+ if "comparison_period" in explicit_fields and config.comparison_period is None:
+ new_form_data["time_compare"] = None
+ if "comparison_type" in explicit_fields and config.comparison_type is None:
+ new_form_data["comparison_type"] = None
+
+ # A Waterfall axis replacement cannot inherit a bucket belonging to the old
+ # temporal subject. Grain omission preserves only while the axis is stable;
+ # explicit null is already handled by the declarative control map above.
+ if (
+ config.chart_type == "waterfall"
+ and existing_form_data.get("x_axis") != new_form_data.get("x_axis")
+ and "time_grain" not in explicit_fields
+ ):
+ new_form_data["time_grain_sqla"] = None
+
+
+def merge_same_viz_form_data(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Preserve saved controls that the typed mapper does not represent.
+
+ The typed MCP surface deliberately models a bounded subset of every Explore
+ control panel. For a replacement within the exact same native ``viz_type``,
+ keys absent from the mapper therefore represent omitted controls and retain
+ their saved values. Mapper output and the chart-specific merge helpers run
+ first and remain authoritative, including explicit empty, false, null, and
+ nested values.
+
+ No generic state crosses a visualization boundary. This prevents query-role
+ keys from the previous plugin (for example ``metric`` or ``groupby``) from
+ leaking into a different plugin whose role contract is unrelated.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if not isinstance(existing_viz_type, str) or existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+
+ _apply_explicit_form_controls(existing_form_data, new_form_data, config)
+
+ for key, value in existing_form_data.items():
+ if key in {"where", "having", "filters"}:
+ # merge_update_form_data already resolved these legacy predicates
+ # into adhoc_filters; restoring them would undo replacement/clear.
+ continue
+ if key == MCP_DASHBOARD_TIME_FILTER_SUBJECT:
+ # merge_update_form_data owns this provenance marker. Its absence
+ # may be an intentional subject clear and must not be undone by the
+ # generic preservation layer.
+ continue
+ if key not in new_form_data:
+ new_form_data[key] = value
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet controls without reinterpreting inherited query roles.
+
+ Saved Explore state may contain SQL dimensions and SQL WHERE/SIMPLE HAVING
+ filters beyond the typed authoring surface. Omitted roles are validated by
+ the native query contract and compilation, not as newly authored physical
+ columns or SIMPLE WHERE filters. Only this validation copy excludes them
+ and native query metadata; compiled and persisted form data stays intact.
+ Explicit replacements, including ``[]``, retain strict typed validation.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ for native_query_key in ("url_params", "extra_form_data", "extra_filters"):
+ validation_data.pop(native_query_key, None)
+ if update_config is None or isinstance(update_config, BulletChartConfig):
+ if update_config is None or update_config.dimensions is None:
+ validation_data.pop("groupby", None)
+ if update_config is None or "filters" not in update_config.model_fields_set:
+ validation_data.pop("adhoc_filters", None)
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ return BulletChartConfig.model_validate(validation_data)
Review Comment:
A labels-only Bullet update that omits `dimensions` works with a saved independent sort (say `orderby: [["Profit", false]]` on a saved `Sales` by `Region` chart), but restating the same `dimensions: [{"name": "Region"}]` fails with `order_by must reference a Bullet dimension or metric output; unknown: Profit`. The merge keeps the inherited sorter, and this validation copy then treats it as newly authored. Could the inherited `orderby` stay out of the strict check unless the update itself supplies `order_by`, as is already done for `groupby` and `adhoc_filters`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4175658079
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1097,364 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = value.split(",")
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str:
+ token = token.strip(_JAVASCRIPT_WHITESPACE)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str:
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", token):
+ token = int(token, 0)
+ elif not re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ token,
+ ):
+ continue
+ try:
+ number = float(token)
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ metric_reference = "".join(
+ "\\" + char if char in ".[]\\" else char for char in model.metric_field
+ )
+ category_field = _unique_bullet_category_field(model.rows)
+ row_field = _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_row", (category_field,)
+ )
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field, row_field)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[row_field] = row_index
+ copied[category_field] = (
+ ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ if model.dimensions
+ else ""
+ )
+ if (
+ range_tooltip_field is not None
+ and containing_range_labels[row_index] is not None
+ ):
+ copied[range_tooltip_field] = containing_range_labels[row_index]
+ values.append(copied)
+
+ # Explore uses indexed rows even when their display labels are identical.
+ category_labels = [row[category_field] for row in values]
+ vega_format = {
+ "SMART_NUMBER": "~s",
+ "SMART_NUMBER_SIGNED": "+~s",
+ }.get(model.y_axis_format, model.y_axis_format)
+ y_encoding = {
+ "field": row_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ "sort": None,
+ "axis": {"labelExpr": f"{json.dumps(category_labels)}[datum.value]"},
+ }
+ tooltip = [
+ {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ },
+ {
+ "field": metric_reference,
+ "type": "quantitative",
+ "title": model.metric_field,
+ "format": vega_format,
+ },
+ ]
+ if range_tooltip_field is not None:
+ tooltip.append(
+ {"field": range_tooltip_field, "type": "nominal", "title": "Range"}
+ )
+ axis_min = min(
+ 0.0,
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ axis_max = max(
Review Comment:
Fixed in 4b2b11b3a1351fae2eaccb2abc914d601aba4a5a. This is introduced by the PR's Bullet-specific renderer, not a pre-existing master path. The maximum receives an iterable, so an empty grouped result with a single range and no markers returns an empty Vega preview. The grouped-empty regression covers both default ranges and the single 100 threshold.
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1888 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+ return _ErrorText(text="; ".join(parts) or None)
+
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
return None
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
Review Comment:
Fixed in 4b2b11b3a1351fae2eaccb2abc914d601aba4a5a. The Treemap/Gauge numeric checks pre-exist on master, but this PR introduces the exact-decimal string projection that makes those checks incompatible. Their numeric consumers accept bounded finite decimal wire strings without changing the response/export values; only rendering projects to binary64. The real object-DataFrame longdouble regression covers result normalization, compilation, and both Vega renderers, retaining the exact wire value. Nonnumeric, underscore, NaN and Infinity strings remain rejected.
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1888 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+ return _ErrorText(text="; ".join(parts) or None)
+
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
return None
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget, *, row_shaped: bool = False
+) -> str | None:
+ """Bound metadata, charging row-shaped indexes to the row-data work budget."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if row_shaped:
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "metadata contains too many total values"
+ else:
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = list.__len__(item)
+ # Chart Data emits one indexname per row. Only that outer array
+ # receives the row limit; containers within an index stay bounded.
+ max_items = (
+ MAX_QUERY_RESULT_ROWS
+ if row_shaped and depth == 0
+ else _MAX_ROW_CONTAINER_ITEMS
+ )
+ if width > max_items:
+ return "metadata contains an oversized array"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(width, mapping=False),
+ metadata=True,
+ ):
+ return reason
+ stack.extend(
+ (list.__getitem__(item, index), depth + 1, False)
+ for index in range(width)
+ )
+ continue
+
+ normalized, reason = _normalize_trusted_scalar(
+ item, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ )
+ if reason is not None:
+ return f"metadata {reason}"
+ if reason := _charge_json_bytes(
+ budget,
+ _normalized_scalar_json_size(
+ normalized, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ ),
+ metadata=True,
+ ):
+ return reason
+ return None
+
+
+def _metadata_failure( # noqa: C901
+ payload: dict[str, Any], label: str
+) -> ChartError | None:
+ """Validate bounded cache and row-count metadata before any consumer."""
+ for key in ("rowcount", "total_rows"):
+ if key in payload and dict.__getitem__(payload, key) is not None:
+ try:
+ bounded_result_row_count(dict.__getitem__(payload, key))
+ except ValueError as ex:
+ return _malformed_result(f"{label} {key} {ex}")
+
+ if "is_cached" in payload:
+ is_cached = dict.__getitem__(payload, "is_cached")
+ if is_cached is not None and type(is_cached) is not bool:
+ return _malformed_result(f"{label} is_cached must be an exact boolean")
+
+ if "cache_key" in payload:
+ cache_key = dict.__getitem__(payload, "cache_key")
+ if cache_key is not None and (
+ type(cache_key) is not str
+ or _bounded_utf8_length(cache_key, _MAX_CACHE_STRING_BYTES) is None
+ ):
+ return _malformed_result(
+ f"{label} cache_key must be a bounded exact string"
+ )
+
+ # ChartData's production schema emits ``cached_dttm``. ``cache_dttm`` was
+ # used by earlier MCP payloads and remains a bounded compatibility alias.
+ # Validate both before cache utilities parse or compare either value.
+ for key in ("cached_dttm", "cache_dttm"):
+ if key not in payload:
+ continue
+ cache_dttm = dict.__getitem__(payload, key)
+ if cache_dttm is None:
+ continue
+ if type(cache_dttm) is str:
+ if _bounded_utf8_length(cache_dttm, _MAX_CACHE_STRING_BYTES) is not None:
+ continue
+ return _malformed_result(f"{label} {key} must be a bounded exact string")
+ if type(cache_dttm) is datetime:
+ tzinfo = cache_dttm.tzinfo
+ if tzinfo is None or any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ continue
+ return _malformed_result(f"{label} {key} has an unsupported timezone")
+ return _malformed_result(
+ f"{label} {key} must be a bounded exact string or datetime"
+ )
+ return None
+
+
+def _normalize_row_value( # noqa: C901
+ value: Any,
+ budget: _ResultBudget,
+ *,
+ temporal_json_numbers: bool = False,
+ preserve_nonfinite_floats: bool = False,
+) -> str | None:
+ """Normalize trusted scalars and return a bounded serialization failure.
+
+ ChartDataCommand materializes a real DataFrame with ``to_dict(records)``, so
+ exact pandas and NumPy scalars can remain in an otherwise valid result. They
+ are converted in place to their canonical JSON-facing builtin values here.
+ Exact containers are inspected through builtin operations only; arbitrary
+ subclasses and scalar hooks remain rejected.
+ """
+ stack: list[
+ tuple[
+ Any,
+ list[Any] | dict[str, Any] | None,
+ int | str | None,
+ int,
+ bool,
+ ]
+ ] = [(value, None, None, 0, False)]
+ active_containers: set[int] = set()
+
+ while stack:
+ item, parent, slot, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "exceeds the nesting depth limit"
+
+ if type(item) is list:
Review Comment:
Fixed in 4b2b11b3a1351fae2eaccb2abc914d601aba4a5a. The unsupported-tuple rejection is introduced by this PR's bounded result walker. Exact builtin tuples are normalized to JSON arrays with the same width, depth, byte and cycle limits; subclasses are not accepted as containers. A real df_to_records regression covers a Tuple-shaped cell, nested tuples and a NumPy scalar, with separate oversized-array and tuple/list-cycle regressions.
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,434 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
Review Comment:
Fixed in 4b2b11b3a1351fae2eaccb2abc914d601aba4a5a. The native reference validator is added by this PR; QueryObject's guarded legacy saved-metric support already exists on master. Native validation recognizes label-only saved metrics while retaining the guard against sqlExpression/aggregate/column dictionaries missing expressionType. The real Table dataset-rebind validator regression accepts Revenue in the replacement dataset and rejects missing saved metrics and malformed adhoc definitions.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,454 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
Review Comment:
Fixed in 4b2b11b3a1351fae2eaccb2abc914d601aba4a5a. This PR adds the Table temporal BASE_AXIS reconstruction, so the missing semantic flag is in scope rather than pre-existing on master. The wrapper preserves isColumnReference=true for semantic-view sources, matching the frontend Table builder. Regressions cover physical and semantic sources, two temporal columns, and a preceding non-temporal dimension; the resulting semantic axis also passes the real semantic mapper's column normalization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4176712378
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,459 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ **(
+ {"isColumnReference": True}
+ if str(form_data.get("datasource", "")).endswith("__semantic_view")
+ else {}
+ ),
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
+ ):
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
+
+
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
+ x_axis = form_data.get("x_axis")
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
+ },
+ }
+ )
+ rolling_type = form_data.get("rolling_type")
+ rolling_columns = (
+ [*offset_map.values(), *offset_map.keys()] if comparison else metric_labels
+ )
+ if rolling_type == "cumsum":
+ chain.append(
+ {
+ "operation": "cum",
+ "options": {
+ "operator": "sum",
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ elif rolling_type in {"sum", "mean", "std"}:
+ chain.append(
+ {
+ "operation": "rolling",
+ "options": {
+ "rolling_type": rolling_type,
+ "window": int(form_data.get("rolling_periods") or 1),
+ "min_periods": int(form_data.get("min_periods") or 0),
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ comparison_type = form_data.get("comparison_type")
+ if comparison and comparison_type != "values":
+ chain.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": list(offset_map.values()),
+ "compare_columns": list(offset_map.keys()),
+ "compare_type": comparison_type,
+ "drop_original_columns": True,
+ },
+ }
+ )
+ if complete_timeseries_contract and form_data.get("contributionMode"):
+ chain.append(
+ {
+ "operation": "contribution",
+ "options": {
+ "orientation": form_data["contributionMode"],
+ "time_shifts": offsets if comparison else [],
+ },
+ }
+ )
+ if comparison:
+ rename: dict[str, str | None] = {}
+ for shifted, metric in offset_map.items():
+ offset = next(
+ (item for item in offsets if shifted.endswith(f"__{item}")), None
+ )
+ source = (
+ shifted
+ if comparison_type == "values"
+ else f"{comparison_type}__{metric}__{shifted}"
+ )
+ rename[source] = f"{metric}, {offset}" if len(metrics) > 1 else offset
+ if rename:
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {"columns": rename, "level": 0, "inplace": True},
+ }
+ )
+ elif (
+ x_label
+ and len(metrics) == 1
+ and (series_labels or len(offsets) > 1)
+ and form_data.get("truncate_metric") is not None
+ and form_data.get("truncate_metric")
+ ):
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {
+ "columns": {metric_labels[0]: None},
+ "level": 0,
+ "inplace": True,
+ },
+ }
+ )
+ if complete_timeseries_contract:
+ x_axis_sort = form_data.get("x_axis_sort")
+ x_axis_sort_asc = form_data.get("x_axis_sort_asc")
+ sortable_labels = [
+ label
+ for label in [
+ x_label,
+ *(
+ _metric_label(metric)
+ for metric in _as_list(form_data.get("metrics"))
+ ),
+ *(
+ _metric_label(metric)
+ for metric in _timeseries_extra_metrics(form_data)
+ ),
+ ]
+ if label
+ ]
if (
- not isinstance(entry, (list, tuple))
- or len(entry) != 2
- or not isinstance(entry[0], str)
- or not entry[0]
- or not isinstance(entry[1], bool)
+ x_axis_sort is not None
+ and x_axis_sort_asc is not None
+ and x_axis_sort in sortable_labels
+ and not _as_list(form_data.get("groupby"))
):
- raise ValueError(
- f"Gantt order_by_cols[{index}] must be [column, ascending_boolean]"
- )
- orderby.append([entry[0], entry[1]])
+ options: dict[str, Any] = {"ascending": x_axis_sort_asc}
+ if x_axis_sort == x_label:
+ options["is_sort_index"] = True
+ else:
+ options["by"] = x_axis_sort
+ chain.append({"operation": "sort", "options": options})
+ chain.append({"operation": "flatten"})
+
+ if complete_timeseries_contract and form_data.get("forecastEnabled") and x_label:
+ x_axis_grain = (
+ x_axis.get("timeGrain")
+ if isinstance(x_axis, dict)
+ and x_axis.get("expressionType") in {"SIMPLE", "SQL"}
+ else None
+ )
+ time_grain = (
+ x_axis_grain
+ or (query.get("extras") or {}).get("time_grain_sqla")
+ or form_data.get("time_grain_sqla")
+ or "P1D"
+ )
+ chain.append(
+ {
+ "operation": "prophet",
+ "options": {
+ "time_grain": time_grain,
+ "periods": int(form_data.get("forecastPeriods", 10)),
+ "confidence_interval": float(
+ form_data.get("forecastInterval", 0.8)
+ ),
+ "yearly_seasonality": form_data.get("forecastSeasonalityYearly"),
+ "weekly_seasonality": form_data.get("forecastSeasonalityWeekly"),
+ "daily_seasonality": form_data.get("forecastSeasonalityDaily"),
+ "index": x_label,
+ },
+ }
+ )
+ return [operator for operator in chain if operator is not None]
- columns: list[Any] = []
- seen: set[str] = set()
- for column in (
- start_time,
- end_time,
- category,
- *series_columns,
- *tooltip_columns,
- *(entry[0] for entry in orderby),
+
+def _mixed_layer_form_data(
+ form_data: dict[str, Any], *, secondary: bool
+) -> dict[str, Any]:
+ """Mirror MixedTimeseries remove/retainFormDataSuffix for one layer."""
+ if not secondary:
+ return {
+ key: value for key, value in form_data.items() if not key.endswith("_b")
+ }
+ layer = {key: value for key, value in form_data.items() if not key.endswith("_b")}
+ for isolated_key in (
+ "metrics",
+ "groupby",
+ "orderby",
+ "limit",
+ "series_limit",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ "order_desc",
+ "truncate_metric",
+ "time_compare",
+ "comparison_type",
+ "resample_method",
+ "resample_rule",
+ "rolling_type",
+ "rolling_periods",
+ "min_periods",
+ "show_empty_columns",
):
- key = utils_json.dumps(column, sort_keys=True, default=str)
- if key not in seen:
- seen.add(key)
- columns.append(column)
- return columns, list(raw_tooltip_metrics), orderby, series_columns
+ if f"{isolated_key}_b" not in form_data:
+ layer.pop(isolated_key, None)
Review Comment:
Fixed in 1b008ec02add137eeaab93a483eaf2a5ea37e00d. Introduced by this PR's Mixed layer reconstruction, not pre-existing on master. The secondary layer now mirrors retainFormDataSuffix: shared controls remain unless an explicit _b value overrides them. Regressions cover shared limit/timeseries_limit_metric/order_desc, suffixed overrides, and shared ordering/roles; explicit secondary analytics overrides remain independent.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,459 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ **(
+ {"isColumnReference": True}
+ if str(form_data.get("datasource", "")).endswith("__semantic_view")
+ else {}
+ ),
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
+ ):
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
+
+
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
+ x_axis = form_data.get("x_axis")
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
Review Comment:
Fixed in 1b008ec02add137eeaab93a483eaf2a5ea37e00d. Introduced by this PR's timeseries post-processing reconstruction, not pre-existing on master. The resample operator preserves fill_time_range when resample_fill_time_range is enabled, matching Explore. A regression executes the real QueryObject/Pandas chain with observations inside the requested period and verifies leading/trailing zero-filled days; disabled filling retains only the observed period.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,532 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip()
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = float(Decimal(stripped))
Review Comment:
Fixed in 1b008ec02add137eeaab93a483eaf2a5ea37e00d. Introduced by this PR's strict Bullet metric parser, not pre-existing on master. Metric strings share the native-control JavaScript numeric grammar and ECMAScript trimming rather than Decimal's permissive grammar. Compile and render-model regressions reject underscore separators, non-ASCII digits, and Python-only whitespace; valid decimal/exponent/radix strings and exact decimal wire values remain supported.
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1897 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget, *, row_shaped: bool = False
+) -> str | None:
+ """Bound metadata, charging row-shaped indexes to the row-data work budget."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if row_shaped:
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "metadata contains too many total values"
+ else:
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = list.__len__(item)
+ # Chart Data emits one indexname per row. Only that outer array
+ # receives the row limit; containers within an index stay bounded.
+ max_items = (
+ MAX_QUERY_RESULT_ROWS
+ if row_shaped and depth == 0
+ else _MAX_ROW_CONTAINER_ITEMS
+ )
+ if width > max_items:
+ return "metadata contains an oversized array"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(width, mapping=False),
+ metadata=True,
+ ):
+ return reason
+ stack.extend(
+ (list.__getitem__(item, index), depth + 1, False)
+ for index in range(width)
+ )
+ continue
+
+ normalized, reason = _normalize_trusted_scalar(
+ item, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
Review Comment:
Fixed in 1b008ec02add137eeaab93a483eaf2a5ea37e00d. Introduced by this PR's bounded metadata walker, not pre-existing on master. Exact builtin tuples are accepted as bounded metadata containers with the same work, byte, width, depth, and cycle accounting as lists; subclasses remain rejected. A real two-dimension Pandas pivot regression validates its tuple indexnames and JSON array serialization, with additional tuple metadata boundary regressions.
##########
superset/mcp_service/semantic_layer/tool/get_table.py:
##########
@@ -366,23 +372,25 @@ def _build_response(
is_builtin: bool,
display_name: str,
query_result: dict[str, Any],
+ data: list[dict[str, Any]],
+ raw_columns: list[str],
+ coltypes: list[int | GenericDataType],
query_duration_ms: int,
warnings: list[str],
temporal_columns: set[str] | None = None,
valid_grains: dict[str, dict[str, str]] | None = None,
) -> GetTableResponse:
"""Format the query result into a GetTableResponse."""
- data = query_result.get("data", [])
- raw_columns = query_result.get("colnames", [])
cache_status = get_cache_status_from_result(
query_result, force_refresh=request.force_refresh
)
+ columns_meta = format_data_columns(data, raw_columns, coltypes)
Review Comment:
Fixed in 1b008ec02add137eeaab93a483eaf2a5ea37e00d. Introduced by this PR: master profiles nonempty responses only once (https://github.com/apache/superset/blob/68f19947a8/superset/mcp_service/semantic_layer/tool/get_table.py#L403-L413). The initial profiling call is restricted to the empty-result branch; nonempty results retain one pass with temporal overrides. Empty/nonempty call-count regressions pass, and all 100 get_table unit tests pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4225950175
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,137 @@ def fallback_vega_lite_preview(
return None
+def _xy_pivot_x_type(values: list[Any]) -> str:
+ """Infer the Vega-Lite x type from every x value, not a character scan.
+
+ The renderer has no column metadata, so text is temporal only when each
+ value parses as an ISO date or datetime; labels such as ``New York`` stay
+ nominal instead of becoming unparseable dates.
+ """
+ present = [value for value in values if value is not None]
+ if not present:
+ return "nominal"
+ if all(
+ isinstance(value, (int, float)) and not isinstance(value, bool)
+ for value in present
+ ):
+ return "quantitative"
+ if all(
+ isinstance(value, (date, datetime))
+ or (isinstance(value, str) and _gantt_temporal_value(value) is not None)
Review Comment:
A grouped XY result whose x values are compact ISO dates (`20250101`, `20250102`) passes this check, because Python's ISO parser accepts that form, so the preview marks x as `temporal` and leaves the strings unchanged. Vega parses temporal strings with `Date.parse`, which returns `NaN` for them, so the bars drop out of an otherwise successful preview. Should the inference only accept the extended ISO forms the renderer can parse, and fall back to `nominal` otherwise?
##########
superset/mcp_service/semantic_layer/tool/get_table.py:
##########
@@ -509,33 +563,38 @@ async def _run_get_table_query(
use_cache=request.use_cache,
force=request.force_refresh,
)
+ extracted = _extract_table_query_result(result)
+ if isinstance(extracted, SemanticLayerError):
+ return extracted
+ query_result, data, raw_columns, coltypes = extracted
query_duration_ms = int((time.time() - start_time) * 1000)
- if not result or "queries" not in result or not result["queries"]:
- return SemanticLayerError.create(
- error="Query returned no results.",
- error_type="EmptyQuery",
- )
-
await ctx.report_progress(5, 5, "Formatting results")
- query_result = result["queries"][0]
response = _build_response(
request,
is_builtin,
resolved.display_name,
query_result,
+ data,
+ raw_columns,
+ coltypes,
query_duration_ms,
resolved.warnings,
resolved.temporal_columns,
resolved.valid_grains,
)
+ if response_failure := response_json_failure(response):
Review Comment:
Nothing exercises this post-format `response_json_failure` guard through `get_table`. The existing malformed-result cases (`test_get_table_maps_invalid_result_without_formatting_hooks`) all reject at source validation, before formatting runs, so deleting this block would leave them passing. Could you add a case with about 255 rows that each carry a distinct 64 KiB `category` cell (valid at the source) and assert that the repeated profiling samples produce a `MalformedQueryResult` error with no data returned?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5502426069
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3936359708
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -323,6 +370,536 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip()
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = float(Decimal(stripped))
+ except (InvalidOperation, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet number format {format_!r} is unsupported by previews",
+ error_type="UnsupportedFormat",
+ ) from ex
+ return format_
+
+
+def _format_bullet_number(format_: str, value: float) -> str:
+ """Format finite Bullet values, including the full binary-float range."""
+ from superset.utils.number_format import format_numeric
+
+ try:
+ return format_numeric(format_, value)
+ except OverflowError:
+ # SMART_NUMBER's significant-digit rounding can overflow a finite float
+ # near DBL_MAX. Scientific repr remains deterministic and informative.
+ if format_ in {"SMART_NUMBER", "SMART_NUMBER_SIGNED"} and math.isfinite(value):
+ prefix = "+" if format_ == "SMART_NUMBER_SIGNED" and value > 0 else ""
+ return prefix + repr(value)
+ raise
+
+
+def _containing_bullet_range_label(
+ measure: float, ranges: list[float], labels: list[str]
+) -> str | None:
+ """Match the frontend's labelled containing-range tooltip selection."""
+ ascending = sorted(zip(ranges, labels, strict=True), key=lambda entry: entry[0])
+ for threshold, label in ascending:
+ if measure <= threshold:
+ return label or None
+ if ascending and ascending[-1][1]:
+ return f"> {ascending[-1][1]}"
+ return None
+
+
+def resolve_bullet_render_model( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> BulletRenderModel:
+ """Resolve and validate every Bullet row and presentation control."""
+ if type(data) is not list:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ for row_index in range(list.__len__(data)):
+ row = list.__getitem__(data, row_index)
+ if type(row) is not dict:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ if dict.__len__(row) > _MAX_BULLET_FIELDS:
+ raise BulletOutputError("Bullet query row exceeds the field limit")
+ for key in dict.keys(row):
+ if type(key) is not str:
+ raise BulletOutputError("Bullet query row keys must be strings")
+ if len(key) > _MAX_BULLET_FIELD_BYTES:
+ raise BulletOutputError("Bullet query row key exceeds the size limit")
+
+ if type(form_data) is not dict:
+ raise BulletOutputError("Bullet form data must be an object")
+
+ metric_label = _form_metric_label(dict.get(form_data, "metric"))
+ if not metric_label:
+ raise BulletOutputError("Bullet metric has no declared result alias")
+ raw_groupby = dict.get(form_data, "groupby")
+ if raw_groupby is None:
+ raw_groupby = []
+ if type(raw_groupby) is not list:
+ raise BulletOutputError("Bullet dimensions must be an array")
+ dimension_labels = [
+ _form_column_label(list.__getitem__(raw_groupby, index))
+ for index in range(list.__len__(raw_groupby))
+ ]
+ if any(not label for label in dimension_labels):
+ raise BulletOutputError("Bullet dimension has no declared result alias")
+
+ if data:
+ first_row = list.__getitem__(data, 0)
+ metric_field = _require_result_field(metric_label, first_row, "metric")
+ dimensions = [
+ _require_result_field(label, first_row, "dimension")
+ for label in dimension_labels
+ ]
+ else:
+ # The frontend accepts empty results. Ungrouped charts retain one
+ # zero-valued measure; grouped charts retain the declared roles but no
+ # categories or rows are fabricated.
+ metric_field = metric_label
+ dimensions = [label for label in dimension_labels if label is not None]
+
+ measures: list[float] = []
+ copied_rows: list[dict[str, Any]] = []
+ for index in range(list.__len__(data)):
+ row = list.__getitem__(data, index)
+ row_metric_field = _require_result_field(
+ metric_label, row, f"metric row {index}"
+ )
+ measure = _bullet_number(
+ dict.__getitem__(row, row_metric_field), index, metric_field
+ )
+ # Reserve every exact output key so the internal Vega category alias
+ # cannot collide with an unselected result field. Unselected values are
+ # deliberately replaced with None rather than converted or serialized.
+ copied: dict[str, Any] = dict.fromkeys(dict.keys(row))
+ copied[metric_field] = measure
+ for label, dimension in zip(dimension_labels, dimensions, strict=True):
+ row_dimension = _require_result_field(label, row, f"dimension row {index}")
+ dimension_value, _ = _bullet_category_value(
+ dict.__getitem__(row, row_dimension), dimension, index
+ )
+ copied[dimension] = dimension_value
+ copied_rows.append(copied)
+ measures.append(measure)
+
+ # The frontend validates/coerces the whole result array but renders only
+ # the first row for an ungrouped aggregate.
+ if not dimensions:
+ copied_rows = copied_rows[:1]
+ measures = measures[:1]
+ if not copied_rows:
+ copied_rows = [{metric_field: 0.0}]
+ measures = [0.0]
+
+ ranges = _strict_bullet_numeric_tokens(dict.get(form_data, "ranges"), "ranges")
+ if not ranges:
+ # Match Bullet/transformProps.ts: the largest measure drives one
+ # qualitative band whose upper threshold is 110% of that measure.
+ ranges = [0.0, max(measures, default=0.0) * 1.1]
+ markers = _strict_bullet_numeric_tokens(dict.get(form_data, "markers"), "markers")
+ marker_lines = _strict_bullet_numeric_tokens(
+ dict.get(form_data, "marker_lines"), "marker lines"
+ )
+ all_numbers = [*measures, *ranges, *markers, *marker_lines]
+ if any(not math.isfinite(value) for value in all_numbers):
+ raise BulletOutputError("Bullet presentation values must be finite")
+
+ show_labels = dict.get(form_data, "show_labels", False)
+ show_legend = dict.get(form_data, "show_legend", False)
+ if type(show_labels) is not bool or type(show_legend) is not bool:
+ raise BulletOutputError("Bullet label and legend controls must be booleans")
+
+ range_labels = _bullet_string_tokens(dict.get(form_data, "range_labels"))
+ marker_labels = _bullet_string_tokens(dict.get(form_data, "marker_labels"))
+ marker_line_labels = _bullet_string_tokens(
+ dict.get(form_data, "marker_line_labels")
+ )
+ for role, labels, values in (
+ ("range", range_labels, ranges),
+ ("marker", marker_labels, markers),
+ ("marker line", marker_line_labels, marker_lines),
+ ):
+ if labels and len(labels) != len(values):
Review Comment:
This rejects a saved Bullet chart when its label list is shorter than its ranges or markers, although `Bullet/transformProps.ts` intentionally renders missing labels as blank or as the formatted value. That makes MCP previews fail for charts that still render in Explore. Could the preview use the same per-index fallback instead of requiring one label per value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5794458808
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5883091053
@rusackas thanks, both addressed in d040008011. For the lookup split, update_chart and update_chart_preview now enter a registry `saved_chart_contract()` scope for the saved (or cached) chart, so the strict rebind guard, `resolve_update_config`, normalization and mapping resolve the same plugin `plugin_for_viz_type()` does even when that type is disabled; other disabled types stay unreachable, so an update still can't switch a chart to one. Previously these updates failed outright in mapping, which the new disabled-Treemap tests reproduced. Gantt's `allows_empty_result` came in with this PR's plugin-contract refactor and isn't on master, so I dropped it: a zero-row Gantt query returns NoData again, with saved and unsaved tests for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5943902572
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5948887206
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4224693206
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,112 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ prefixes = {
+ spelling
+ for label in metric_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
Review Comment:
Confirmed: with one compared metric the rename step names shifted series by the bare offset (`1 year ago, East`), so the metric-prefix filter dropped them. Fixed in 42e861005ca93f4d00cb60f87478eeff98fd32eb: when `_time_comparison` holds and there is one metric, field discovery also accepts each `time_compare` offset as a series prefix (multi-metric renames already keep the `revenue, 1 year ago, …` prefix). New `test_xy_preview_renders_offset_named_comparison_series[values|difference]` runs the real post-processing chain, unescapes columns like the query context processor, and asserts every non-x column, including both `1 year ago, …` series, is folded; both cases fail without the fix.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,112 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ prefixes = {
+ spelling
+ for label in metric_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(prefix + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(prefix + "__")
+ for prefix in prefixes
+ )
+ ]
+ if not fields and len(metric_labels) == 1 and form_data.get("truncate_metric"):
+ # A single truncated metric drops its label from the pivoted column
+ # names, so every non-x-axis column is one category series.
+ fields = [field for field in data[0] if field != x_axis]
+ if not fields:
+ return None
+ sample = data[0][x_axis]
+ x_type = (
+ "temporal"
+ if isinstance(sample, str) and any(char in sample for char in "-/: ")
+ else "quantitative"
+ if isinstance(sample, (int, float))
+ else "nominal"
+ )
+ return VegaLitePreview(
+ specification={
+ "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
+ "data": {"values": data},
+ "transform": [
+ {
+ "fold": [
+ "".join(
+ "\\" + char if char in ".[]\\" else char for char in field
+ )
+ for field in fields
+ ],
+ "as": ["__mcp_xy_series", "__mcp_xy_value"],
+ }
+ ],
+ "mark": mark,
+ "encoding": {
+ "x": {"field": x_axis, "type": x_type, "title": x_axis},
Review Comment:
Confirmed. Fixed in 42e861005ca93f4d00cb60f87478eeff98fd32eb: the x encoding and the x tooltip field use the same `.`/`[`/`]`/`\` escaping as the folded series (one `vega_field` helper), while titles keep the raw name. New `test_xy_preview_escapes_dotted_x_axis_field` uses a real `orders.date` x-axis result and asserts `orders\\.date` in both the encoding and tooltip; it fails without the fix.
##########
superset/mcp_service/chart/tool/update_chart_preview.py:
##########
@@ -208,11 +245,27 @@ def update_chart_preview( # noqa: C901
or (previous_form_data or {}).get("datasource_id")
or ""
).split("__", 1)[0]
- plugin = get_registry().get(config.chart_type, include_disabled=True)
+ # The cached chart keeps its plugin's contract for the whole
+ # update, even when its chart type is disabled for new charts.
+ contract_scope.enter_context(
+ saved_chart_contract((previous_form_data or {}).get("viz_type"))
+ )
+ plugin = get_registry().get(config.chart_type)
dataset_rebind = previous_datasource != str(dataset.id) and (
bool(previous_datasource)
or bool(plugin and plugin.unbound_form_data_is_rebind)
)
+ if (
+ dataset_rebind
+ and previous_form_data
+ and not (plugin is not None and plugin.strict_dataset_rebind)
+ ):
+ # Match saved-chart rebinds: retain only references that resolve
+ # against the replacement dataset before resolving omitted roles.
+ previous_form_data = _prune_inherited_query_state(
Review Comment:
Confirmed: the list-only check let a scalar saved `groupby` survive the prune. Fixed in 329591907772d8dd7fe6b0e5b3ac13d34ba6ba7f: `_inherited_state_invalid_keys` reads every inherited column-list role through `_ensure_reference_list` (the `ensureIsArray` rule), so a scalar value is checked as a one-item list for all charts, not only Bullet. New `test_bullet_rebind_prunes_scalar_saved_hierarchy` drives `_prune_inherited_query_state` (the helper `update_chart_preview` calls) against a replacement dataset: `groupby: "OldRegion"` is dropped and `groupby: "Region"` is kept; the drop case fails without the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5867650649
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5841583181
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-6003164465
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4182465186
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,464 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ temporal_pairs: list[tuple[str, str]] = []
+ inert_subjects: set[str] = set()
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ if not isinstance(comparator, str) or not comparator:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs a range"
+ )
+ if comparator.casefold() == "no filter":
+ inert_subjects.add(subject)
+ else:
+ temporal_pairs.append((subject, comparator))
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ if len(temporal_pairs) > 1:
+ raise ValueError(
+ "Multiple active native temporal filters cannot be represented by "
+ "a single temporal_column/time_range pair"
+ )
+ if temporal_pairs:
+ subject, comparator = temporal_pairs[0]
+ if data.get("temporal_column") not in (None, subject) or data.get(
+ "time_range"
+ ) not in (None, "No filter", comparator):
+ raise ValueError(
+ "Native temporal filter conflicts with temporal_column/time_range"
+ )
+ data["temporal_column"] = subject
+ data["time_range"] = comparator
+ elif len(inert_subjects) == 1:
+ data.setdefault("temporal_column", next(iter(inert_subjects)))
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ from superset.mcp_service.chart.preview_utils import (
+ _bullet_numeric_control_tokens,
+ )
+
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ # Saved Explore controls are lenient; newly authored typed lists
+ # still undergo the strict List[float] validation below.
+ for control in ("ranges", "markers", "marker_lines"):
+ if control in data:
+ data[control] = _bullet_numeric_control_tokens(
+ data[control], control
+ )
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
Review Comment:
A labels-only update of a saved Bullet grouped by `{"expressionType":"SQL","sqlExpression":"UPPER(region)","label":"RegionUpper"}` fails here: the preserved native dimension reaches `ColumnRef` unchanged, which rejects its native expression fields, although Explore supports this grouping. Could an omitted inherited hierarchy be validated through the native query contract instead of being reinterpreted as newly supplied physical-column dimensions?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,462 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
Review Comment:
A saved Bullet using the standard `DURATION` or `MEMORY_BINARY` formatter now fails `get_chart_data`, including CSV/Excel export: this render model rejects those frontend presets before returning the raw rows. Could raw row validation avoid preview-formatter checks so valid native presentation choices do not make existing chart data unreadable?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,770 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop saved sorts that no longer target a final exact query output."""
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs = {
+ name for name in new_form_data.get("groupby") or [] if isinstance(name, str)
+ }
+ metrics = new_form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ for metric in [new_form_data.get("metric"), *metrics]:
+ label = metric.get("label") if isinstance(metric, Mapping) else metric
+ if isinstance(label, str):
+ outputs.add(label)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ if isinstance(target, Mapping):
+ target = target.get("label") or target.get("metric_name")
+ if isinstance(target, str) and target not in outputs:
+ continue
+ retained.append(entry)
Review Comment:
Replacing `SUM(revenue)` with `SUM(profit)` while keeping the label `Amount` retains the old metric dictionary in `orderby`, so expression-based ordering still ranks by revenue (or fails after rebinding to a dataset without revenue). Could inherited metric-output sorts be rebound to the final metric object rather than retaining the old expression on label equality alone?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1739,557 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ # Native extractQueryFields excludes empty-string column references.
+ table_columns = [
+ column
+ for column in _as_list(
+ form_data.get("all_columns") if raw_mode else form_data.get("groupby")
+ )
+ if column != ""
+ ]
+ table_metrics = [] if raw_mode else _as_list(form_data.get("metrics"))
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ offsets = _table_time_offsets(form_data, query)
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if offsets:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
Review Comment:
For a Table with `metrics=["sales"]`, `percent_metrics=["profit"]` and a time comparison, this compares both metrics and drops the original profit columns; the native builder compares only sales and retains profit alongside its contribution columns. Could comparison sources use the ordinary metric list rather than the union with percentage-only metrics, with a distinct-metric regression asserting the returned columns?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,770 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop saved sorts that no longer target a final exact query output."""
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs = {
+ name for name in new_form_data.get("groupby") or [] if isinstance(name, str)
+ }
+ metrics = new_form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ for metric in [new_form_data.get("metric"), *metrics]:
+ label = metric.get("label") if isinstance(metric, Mapping) else metric
+ if isinstance(label, str):
+ outputs.add(label)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ if isinstance(target, Mapping):
+ target = target.get("label") or target.get("metric_name")
+ if isinstance(target, str) and target not in outputs:
+ continue
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if isinstance(existing_viz_type, str) and existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+def _currency_form_value(value: CurrencyFormat | None) -> dict[str, str] | None:
+ """Return the native value for an explicitly supplied currency control."""
+ return value.to_form_data() if value is not None else None
+
+
+def _column_names(value: Sequence[ColumnRef] | None) -> list[str | None] | None:
+ """Return a native column-name list while retaining an explicit null."""
+ return [column.name for column in value] if value is not None else None
+
+
+def _table_sort_value(value: Sequence[str | SortByConfig] | None) -> list[str] | None:
+ """Return the native Table sort control for an explicit typed value."""
+ if value is None:
+ return None
+ return [
+ json.dumps(
+ [entry.column, entry.ascending]
+ if isinstance(entry, SortByConfig)
+ else [entry, False]
+ )
+ for entry in value
+ ]
+
+
+def _table_column_config_value(value: Any) -> dict[str, Any] | None:
+ """Return Table column config without losing an explicit null or empty map."""
+ if value is None:
+ return None
+ return {
+ label: column.model_dump(by_alias=True, exclude_unset=True)
+ for label, column in value.items()
+ }
+
+
+# Mappers intentionally omit optional controls so fresh charts use the frontend
+# defaults. During a same-viz update, however, an explicitly supplied false,
+# null, or empty value must block preservation of the saved native key. Keep the
+# typed-to-native relationship declarative so every update path shares it.
+_FormValueConverter = Callable[[Any], Any]
+_FormControlMap = dict[str, tuple[str, _FormValueConverter]]
+
+_COMMON_EXPLICIT_FORM_CONTROLS: _FormControlMap = {
+ "color_scheme": ("color_scheme", lambda value: value),
+ "currency_format": ("currency_format", _currency_form_value),
+ "show_value": ("show_value", lambda value: value),
+}
+
+_CHART_EXPLICIT_FORM_CONTROLS: dict[str, _FormControlMap] = {
+ "table": {
+ "sort_by": ("order_by_cols", _table_sort_value),
+ "column_config": ("column_config", _table_column_config_value),
+ },
+ "xy": {
+ "group_by": ("groupby", _column_names),
+ "series_limit": ("series_limit", lambda value: value),
+ "stacked": ("stack", lambda value: "Stack" if value else None),
+ "orientation": ("orientation", lambda value: value),
+ "legend_orientation": ("legendOrientation", lambda value: value),
+ "x_axis_time_format": ("x_axis_time_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "mixed_timeseries": {
+ "group_by": ("groupby", _column_names),
+ "group_by_secondary": ("groupby_b", _column_names),
+ "currency_format_secondary": (
+ "currency_format_secondary",
+ _currency_form_value,
+ ),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "waterfall": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "big_number": {
+ "subheader": ("subheader", lambda value: value),
+ "y_axis_format": ("y_axis_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "compare_lag": ("compare_lag", lambda value: value),
+ "time_format": ("time_format", lambda value: value),
+ "aggregation": ("aggregation", lambda value: value),
+ },
+ "handlebars": {
+ "style_template": ("styleTemplate", lambda value: value),
+ "columns": ("all_columns", _column_names),
+ "groupby": ("groupby", _column_names),
+ "metrics": ("metrics", _column_names),
+ },
+ "pivot_table": {
+ "date_format": ("date_format", lambda value: value),
+ },
+ "interactive_pivot": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "series_limit": ("series_limit", lambda value: value),
+ "date_format": ("date_format", lambda value: value),
+ "column_sort": ("colOrder", lambda value: value),
+ },
+}
+
+
+def _apply_explicit_form_controls( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply typed controls whose mapper omission represents a native clear."""
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ controls = {
+ **_COMMON_EXPLICIT_FORM_CONTROLS,
+ **_CHART_EXPLICIT_FORM_CONTROLS.get(config.chart_type, {}),
+ }
+ for field_name, (native_key, convert) in controls.items():
+ if field_name in explicit_fields:
+ converted = convert(getattr(config, field_name))
+ is_clear = (
+ converted is None
+ or converted is False
+ or converted == ""
+ or converted in ([], {})
+ )
+ if is_clear:
+ new_form_data[native_key] = converted
+
+ axis_controls = {
+ "xy": (
+ ("x_axis", "x_axis_title", "x_axis_format", None),
+ ("y_axis", "y_axis_title", "y_axis_format", "logAxis"),
+ ),
+ "mixed_timeseries": (
+ ("x_axis", "xAxisTitle", "x_axis_time_format", None),
+ ("y_axis", "yAxisTitle", "y_axis_format", "logAxis"),
+ (
+ "y_axis_secondary",
+ "yAxisTitleSecondary",
+ "y_axis_format_secondary",
+ "logAxisSecondary",
+ ),
+ ),
+ }
+ if config.chart_type in axis_controls:
+ for field_name, title_key, format_key, scale_key in axis_controls[
+ config.chart_type
+ ]:
+ if field_name not in explicit_fields:
+ continue
+ axis = getattr(config, field_name)
+ if axis is None:
+ new_form_data[title_key] = None
+ new_form_data[format_key] = None
+ if scale_key:
+ new_form_data[scale_key] = None
+ continue
+ axis_fields = set(axis.model_fields_set)
+ if "title" in axis_fields:
+ new_form_data[title_key] = axis.title
+ if "format" in axis_fields:
+ new_form_data[format_key] = axis.format
+ if scale_key and "scale" in axis_fields:
+ new_form_data[scale_key] = (
+ None if axis.scale is None else axis.scale == "log"
+ )
+
+ if config.chart_type == "xy" and "legend" in explicit_fields:
+ legend = config.legend
+ if legend is None:
+ new_form_data["show_legend"] = None
+ new_form_data["legendOrientation"] = None
+ else:
+ legend_fields = set(legend.model_fields_set)
+ if "show" in legend_fields:
+ new_form_data["show_legend"] = legend.show
+ if "position" in legend_fields:
+ new_form_data["legendOrientation"] = legend.position
+
+ if config.chart_type == "interactive_pivot":
+ if "temporal_column" in explicit_fields and config.temporal_column is None:
+ new_form_data["granularity_sqla"] = None
+ new_form_data["temporal_columns_lookup"] = None
+ if (
+ "series_limit_metric" in explicit_fields
+ and config.series_limit_metric is None
+ ):
+ new_form_data["series_limit_metric"] = None
+ if "comparison_period" in explicit_fields and config.comparison_period is None:
+ new_form_data["time_compare"] = None
+ if "comparison_type" in explicit_fields and config.comparison_type is None:
+ new_form_data["comparison_type"] = None
+
+ # A Waterfall axis replacement cannot inherit a bucket belonging to the old
+ # temporal subject. Grain omission preserves only while the axis is stable;
+ # explicit null is already handled by the declarative control map above.
+ if (
+ config.chart_type == "waterfall"
+ and existing_form_data.get("x_axis") != new_form_data.get("x_axis")
+ and "time_grain" not in explicit_fields
+ ):
+ new_form_data["time_grain_sqla"] = None
+
+
+def merge_same_viz_form_data(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Preserve saved controls that the typed mapper does not represent.
+
+ The typed MCP surface deliberately models a bounded subset of every Explore
+ control panel. For a replacement within the exact same native ``viz_type``,
+ keys absent from the mapper therefore represent omitted controls and retain
+ their saved values. Mapper output and the chart-specific merge helpers run
+ first and remain authoritative, including explicit empty, false, null, and
+ nested values.
+
+ No generic state crosses a visualization boundary. This prevents query-role
+ keys from the previous plugin (for example ``metric`` or ``groupby``) from
+ leaking into a different plugin whose role contract is unrelated.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if not isinstance(existing_viz_type, str) or existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+
+ _apply_explicit_form_controls(existing_form_data, new_form_data, config)
+
+ for key, value in existing_form_data.items():
+ if key == MCP_DASHBOARD_TIME_FILTER_SUBJECT:
+ # merge_update_form_data owns this provenance marker. Its absence
+ # may be an intentional subject clear and must not be undone by the
+ # generic preservation layer.
+ continue
+ if key not in new_form_data:
+ new_form_data[key] = value
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet state without reclassifying preserved filters.
+
+ The typed Bullet surface intentionally creates only SIMPLE WHERE filters,
+ while saved Explore state may legitimately contain SQL WHERE or SIMPLE
+ HAVING filters. When an update omitted ``filters``, those native objects
+ came from the saved state and are validated by the form-data/query layer;
+ removing them only from this schema-validation copy avoids pretending they
+ were newly supplied typed filters. Explicit filter replacements, including
+ ``[]``, still take the strict native-to-typed path.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ preserves_native_filters = update_config is None or (
+ isinstance(update_config, BulletChartConfig)
+ and "filters" not in update_config.model_fields_set
+ )
+ if preserves_native_filters:
+ validation_data.pop("adhoc_filters", None)
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ return BulletChartConfig.model_validate(validation_data)
Review Comment:
A saved Bullet with `url_params: {"region":"EU"}` now fails even a labels-only update: the merge preserves this key, but `BulletChartConfig` rejects it as unknown before compilation. Could recognized native query metadata be excluded from the typed-validation copy while remaining in the compiled and persisted form data?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-6012481020
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4149958653
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -1441,10 +1420,26 @@ def _write_excel_headers(ws: Any, columns: List[str]) -> None:
ws.cell(row=1, column=idx, value=col)
+def _excel_scalar(value: Any) -> Any:
+ """Project a validated result scalar to values supported by Excel writers."""
+ if value is None:
+ return ""
+ if type(value) is UUID:
+ return UUID.__str__(value)
+ if type(value) is list or type(value) is dict:
+ return str(value)
+ return value
+
+
def _write_excel_data(ws: Any, data: List[Dict[str, Any]], columns: List[str]) -> None:
"""Write data to Excel worksheet."""
for row_idx, row in enumerate(data, 2):
for col_idx, col in enumerate(columns, 1):
+ ws.cell(
+ row=row_idx,
+ column=col_idx,
+ value=_excel_scalar(row.get(col, "")),
Review Comment:
Fixed in fa297506b5c3dbbbc383135c5c388ac7c018af2c: removed the second worksheet write, making _excel_scalar the sole projection for each cell. Non-finite floats are projected to CSV-style text in that shared helper. The new real-workbook regression runs both openpyxl and xlsxwriter with UUIDs, NaN, positive/negative infinity, NULL, lists, dictionaries, and finite numbers.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1778,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
Review Comment:
Fixed in fa297506b5c3dbbbc383135c5c388ac7c018af2c: the shared explicit-empty filter merge also removes the generated temporal-binding marker before provenance validation. Regression coverage exercises save, preview-first, and the cached-preview merge pipeline; omission still preserves the saved binding.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +407,532 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip()
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = float(Decimal(stripped))
+ except (InvalidOperation, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet number format {format_!r} is unsupported by previews",
+ error_type="UnsupportedFormat",
+ ) from ex
+ return format_
+
+
+def _format_bullet_number(format_: str, value: float) -> str:
+ """Format finite Bullet values, including the full binary-float range."""
+ from superset.utils.number_format import format_numeric
+
+ try:
+ return format_numeric(format_, value)
+ except OverflowError:
+ # SMART_NUMBER's significant-digit rounding can overflow a finite float
+ # near DBL_MAX. Scientific repr remains deterministic and informative.
+ if format_ in {"SMART_NUMBER", "SMART_NUMBER_SIGNED"} and math.isfinite(value):
+ prefix = "+" if format_ == "SMART_NUMBER_SIGNED" and value > 0 else ""
+ return prefix + repr(value)
+ raise
+
+
+def _containing_bullet_range_label(
+ measure: float, ranges: list[float], labels: list[str]
+) -> str | None:
+ """Match the frontend's labelled containing-range tooltip selection."""
+ ascending = sorted(
+ (
+ (value, labels[index] if index < len(labels) else "")
+ for index, value in enumerate(ranges)
+ ),
+ key=lambda entry: entry[0],
+ )
+ for threshold, label in ascending:
+ if measure <= threshold:
+ return label or None
+ if ascending and ascending[-1][1]:
+ return f"> {ascending[-1][1]}"
+ return None
+
+
+def resolve_bullet_render_model( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> BulletRenderModel:
+ """Resolve and validate every Bullet row and presentation control."""
+ if type(data) is not list:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ for row_index in range(list.__len__(data)):
+ row = list.__getitem__(data, row_index)
+ if type(row) is not dict:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ if dict.__len__(row) > _MAX_BULLET_FIELDS:
+ raise BulletOutputError("Bullet query row exceeds the field limit")
+ for key in dict.keys(row):
+ if type(key) is not str:
+ raise BulletOutputError("Bullet query row keys must be strings")
+ if len(key) > _MAX_BULLET_FIELD_BYTES:
+ raise BulletOutputError("Bullet query row key exceeds the size limit")
+
+ if type(form_data) is not dict:
+ raise BulletOutputError("Bullet form data must be an object")
+
+ metric_label = _form_metric_label(dict.get(form_data, "metric"))
+ if not metric_label:
+ raise BulletOutputError("Bullet metric has no declared result alias")
+ raw_groupby = dict.get(form_data, "groupby")
+ if raw_groupby is None:
+ raw_groupby = []
+ if type(raw_groupby) is not list:
+ raise BulletOutputError("Bullet dimensions must be an array")
+ dimension_labels = [
+ _form_column_label(list.__getitem__(raw_groupby, index))
+ for index in range(list.__len__(raw_groupby))
+ ]
+ if any(not label for label in dimension_labels):
+ raise BulletOutputError("Bullet dimension has no declared result alias")
+
+ if data:
+ first_row = list.__getitem__(data, 0)
+ metric_field = _require_result_field(metric_label, first_row, "metric")
+ dimensions = [
+ _require_result_field(label, first_row, "dimension")
+ for label in dimension_labels
+ ]
+ else:
+ # The frontend accepts empty results. Ungrouped charts retain one
+ # zero-valued measure; grouped charts retain the declared roles but no
+ # categories or rows are fabricated.
+ metric_field = metric_label
+ dimensions = [label for label in dimension_labels if label is not None]
+
+ measures: list[float] = []
+ copied_rows: list[dict[str, Any]] = []
+ for index in range(list.__len__(data)):
+ row = list.__getitem__(data, index)
+ row_metric_field = _require_result_field(
+ metric_label, row, f"metric row {index}"
+ )
+ measure = _bullet_number(
+ dict.__getitem__(row, row_metric_field), index, metric_field
+ )
+ # Reserve every exact output key so the internal Vega category alias
+ # cannot collide with an unselected result field. Unselected values are
+ # deliberately replaced with None rather than converted or serialized.
+ copied: dict[str, Any] = dict.fromkeys(dict.keys(row))
+ copied[metric_field] = measure
+ for label, dimension in zip(dimension_labels, dimensions, strict=True):
+ row_dimension = _require_result_field(label, row, f"dimension row {index}")
+ dimension_value, _ = _bullet_category_value(
+ dict.__getitem__(row, row_dimension), dimension, index
+ )
+ copied[dimension] = dimension_value
+ copied_rows.append(copied)
+ measures.append(measure)
+
+ # The frontend validates/coerces the whole result array but renders only
+ # the first row for an ungrouped aggregate.
+ if not dimensions:
+ copied_rows = copied_rows[:1]
+ measures = measures[:1]
+ if not copied_rows:
+ copied_rows = [{metric_field: 0.0}]
+ measures = [0.0]
+
+ ranges = _strict_bullet_numeric_tokens(dict.get(form_data, "ranges"), "ranges")
Review Comment:
Fixed in fa297506b5c3dbbbc383135c5c388ac7c018af2c: native Bullet controls drop non-numeric and NaN tokens, matching the frontend tokenizer, while retaining size/item bounds and rejecting infinities. Saved native form-data schema adaptation uses the same parser, so unrelated updates no longer reject these controls and preserve their original strings. Newly authored typed numeric lists remain strict. Tests cover ranges, markers, marker lines, all-invalid/default-range behavior, unrelated updates, and saved/cached ASCII and Vega-Lite calls.
##########
superset/mcp_service/chart/validation/dataset_validator.py:
##########
@@ -524,6 +543,25 @@ def get_canonical_column_name(
The canonical column name from the dataset, or the original name
if no match is found.
"""
+ names = [col["name"] for col in dataset_context.available_columns]
+ names.extend(metric["name"] for metric in dataset_context.available_metrics)
Review Comment:
Fixed in fa297506b5c3dbbbc383135c5c388ac7c018af2c: removed the combined column/metric lookup that shadowed the namespace-aware resolver. Physical columns are resolved first (exact or unique case-insensitive), then saved metrics only if no physical column matches. Explicit saved-metric normalization still uses its separate namespace. Regression cases cover Revenue/REVENUE and lowercase, uppercase, and exact references; ambiguous physical-column tests remain in place.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1373,15 +1406,32 @@ def _bind_dashboard_time_range_filter( # noqa: C901
if _uses_mapper_owned_temporal_binding(form_data, dataset_id):
return
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ if _clears_temporal_subject(config, explicit_fields):
Review Comment:
Fixed in fa297506b5c3dbbbc383135c5c388ac7c018af2c: an explicit null temporal column emits a None marker so the overlay distinguishes clearing from omission. The following provenance merge removes the generated binding without dropping user filters. Tests cover the mapper tombstone and save, preview-first, and cached-preview merge behavior.
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,444 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels must align with their values."
Review Comment:
Fixed in fa297506b5c3dbbbc383135c5c388ac7c018af2c: reworded the general hint to say that labels are optional and missing marker labels use formatted values. It no longer claims matching label counts are required. The regression checks this guidance and validates short and extra label arrays.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4149960129
##########
tests/unit_tests/mcp_service/chart/test_bullet_chart.py:
##########
@@ -0,0 +1,3786 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Product-path coverage for typed ECharts Bullet MCP support."""
+
+import math
+from datetime import date, datetime, time, timedelta, timezone, tzinfo
+from decimal import Decimal
+from enum import Enum, IntEnum, StrEnum
+from types import SimpleNamespace
+from typing import Any
+from unittest.mock import AsyncMock, MagicMock, patch
+from uuid import UUID
+from zoneinfo import ZoneInfo
+
+import numpy as np
+import pandas as pd
+import pytest
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.zoneinfo import get_zonefile_instance
+from pydantic import TypeAdapter, ValidationError
+
+from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+)
+from superset.mcp_service.chart.chart_utils import (
+ analyze_chart_capabilities,
+ map_bullet_config,
+ map_config_to_form_data,
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ merge_update_form_data,
+ validate_merged_bullet_form_data,
+)
+from superset.mcp_service.chart.compile import _compile_chart
+from superset.mcp_service.chart.plugins.bullet import BulletChartPlugin
+from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_preview_from_data,
+ _generate_vega_lite_preview_from_data,
+ _javascript_number_string,
+ BulletOutputError,
+ generate_preview_from_form_data,
+ resolve_bullet_render_model,
+)
+from superset.mcp_service.chart.query_result import (
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+)
+from superset.mcp_service.chart.schemas import (
+ ASCIIPreview,
+ BulletChartConfig,
+ ChartConfig,
+ ChartError,
+ ChartInfo,
+ DataColumn,
+ GenerateChartRequest,
+ GetChartPreviewRequest,
+ UpdateChartPreviewRequest,
+ UpdateChartRequest,
+ VegaLitePreview,
+ XYChartConfig,
+)
+from superset.mcp_service.chart.tool.generate_chart import generate_chart
+from superset.mcp_service.chart.tool.get_chart_data import (
+ _candidates_single_numeric,
+ _VIZ_CATEGORY,
+)
+from superset.mcp_service.chart.tool.get_chart_preview import (
+ ASCIIPreviewStrategy,
+ TablePreviewStrategy,
+ VegaLitePreviewStrategy,
+)
+from superset.mcp_service.chart.tool.get_chart_type_schema import (
+ _get_chart_type_schema_impl,
+ VALID_CHART_TYPES,
+)
+from superset.mcp_service.chart.tool.update_chart import (
+ _build_preview_form_data,
+ _build_update_payload,
+ update_chart,
+)
+from superset.mcp_service.chart.tool.update_chart_preview import update_chart_preview
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+)
+from superset.mcp_service.chart.validation.pipeline import ValidationPipeline
+from superset.mcp_service.common.error_schemas import DatasetContext
+from superset.utils.json import json_int_dttm_ser
+
+
+def _reject_scalar_conversion(*_args: object, **_kwargs: object) -> Any:
+ raise AssertionError("hostile query scalar method must not run")
+
+
+class _PathHostileStr(str):
+ __getitem__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _PathHostileEnum(str, Enum):
+ FAILED = "warehouse unavailable"
+
+ @property
+ def value(self) -> str:
+ """Reject the public descriptor while preserving Enum's stored value."""
+ return _reject_scalar_conversion()
+
+ __getitem__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileInt(int):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileFloat(float):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileStr(str):
+ __str__ = _reject_scalar_conversion
+ strip = _reject_scalar_conversion
+
+
+class _OutputHostileDecimal(Decimal):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputSafeIntEnum(IntEnum):
+ VALUE = 12
+
+
+class _OutputSafeStrEnum(StrEnum):
+ VALUE = "12.5"
+
+
+_OutputSafeIntEnum.__float__ = _reject_scalar_conversion # type: ignore[method-assign]
+_OutputSafeIntEnum.__str__ = _reject_scalar_conversion # type: ignore[method-assign]
+_OutputSafeStrEnum.__float__ = _reject_scalar_conversion # type: ignore[attr-defined]
+_OutputSafeStrEnum.__str__ = _reject_scalar_conversion # type: ignore[method-assign]
+
+
+def _simple_metric(name: str = "revenue") -> dict[str, str]:
+ return {"name": name, "aggregate": "SUM"}
+
+
+def _tool_user() -> SimpleNamespace:
+ return SimpleNamespace(id=1, username="admin", roles=[], groups=[])
+
+
+def _orm_dataset() -> SimpleNamespace:
+ def column(
+ name: str, type_: str, *, temporal: bool = False, numeric: bool = False
+ ) -> SimpleNamespace:
+ return SimpleNamespace(
+ column_name=name,
+ type=type_,
+ is_temporal=temporal,
+ is_numeric=numeric,
+ is_dttm=temporal,
+ python_date_format=None,
+ )
+
+ return SimpleNamespace(
+ id=7,
+ table_name="sales",
+ schema=None,
+ main_dttm_col="OrderDate",
+ database=SimpleNamespace(database_name="main", db_engine_spec=None),
+ columns=[
+ column("Revenue", "NUMERIC", numeric=True),
+ column("Region", "VARCHAR"),
+ column("Team", "VARCHAR"),
+ column("Status", "VARCHAR"),
+ column("OrderDate", "TIMESTAMP", temporal=True),
+ column("EventDate", "TIMESTAMP", temporal=True),
+ ],
+ metrics=[
+ SimpleNamespace(
+ metric_name="SavedRevenue",
+ expression="SUM(Revenue)",
+ description=None,
+ )
+ ],
+ )
+
+
+def test_bullet_discriminated_union_uses_exact_tag() -> None:
+ config = TypeAdapter(ChartConfig).validate_python(
+ {"chart_type": "bullet", "metric": _simple_metric()}
+ )
+ assert isinstance(config, BulletChartConfig)
+ with pytest.raises(ValidationError):
+ TypeAdapter(ChartConfig).validate_python(
+ {"chart_type": "bullet_chart", "metric": _simple_metric()}
+ )
+
+
+def test_bullet_equal_dimension_aliases_are_order_independent_and_round_trip() -> None:
+ for payload in (
+ {
+ "dimensions": [{"name": "Region", "label": "Market"}, "Team"],
+ "groupby": ["Region", {"column_name": "Team"}],
+ },
+ {
+ "groupby": ["Region", {"column": "Team"}],
+ "dimensions": [{"name": "Region", "label": "Market"}, "Team"],
+ },
+ ):
+ config = BulletChartConfig.model_validate(
+ {"metric": _simple_metric(), **payload}
+ )
+ assert [dimension.name for dimension in config.dimensions or []] == [
+ "Region",
+ "Team",
+ ]
+ mapped = map_bullet_config(config)
+ assert mapped["groupby"] == ["Region", "Team"]
+ round_trip = BulletChartConfig.model_validate(mapped)
+ assert [dimension.name for dimension in round_trip.dimensions or []] == [
+ "Region",
+ "Team",
+ ]
+
+
+def test_bullet_dimension_aliases_require_exact_physical_identity() -> None:
+ """Schema-only alias resolution must not merge quoted case variants."""
+ with pytest.raises(ValidationError, match="Conflicting Bullet dimension aliases"):
+ BulletChartConfig.model_validate(
+ {
+ "metric": _simple_metric(),
+ "dimensions": ["Region"],
+ "groupby": ["region"],
+ }
+ )
+
+
+@pytest.mark.parametrize(
+ "request_payload",
+ [
+ {"dataset_id": 7},
+ {"identifier": 9},
+ {"dataset_id": 7, "form_data_key": "preview"},
+ ],
+)
+@pytest.mark.parametrize("reverse", [False, True])
+def test_bullet_request_models_reject_conflicting_dimension_aliases(
+ request_payload: dict[str, object], reverse: bool
+) -> None:
+ aliases = [
+ ("dimensions", [{"name": "Region"}, {"name": "Team"}]),
+ ("groupby", ["Team", "Region"]),
+ ]
+ if reverse:
+ aliases.reverse()
+ config = {"chart_type": "bullet", "metric": _simple_metric(), **dict(aliases)}
+ payload = {**request_payload, "config": config}
+ request_type = (
+ UpdateChartRequest
+ if "identifier" in request_payload
+ else (
+ UpdateChartPreviewRequest
+ if "form_data_key" in request_payload
+ else GenerateChartRequest
+ )
+ )
+ with pytest.raises(ValidationError, match="Conflicting Bullet dimension aliases"):
+ request_type.model_validate(payload)
+
+
+@pytest.mark.parametrize(
+ "metric",
+ [
+ {"name": "revenue", "aggregate": "SUM", "label": "Revenue"},
+ {"name": "saved_revenue", "saved_metric": True},
+ {"sql_expression": "SUM(revenue) / COUNT(*)", "label": "Average"},
+ ],
+)
+def test_bullet_accepts_simple_saved_and_sql_metrics(metric: dict[str, object]) -> None:
+ config = BulletChartConfig(metric=metric)
+ form_data = map_bullet_config(config)
+ assert form_data["viz_type"] == "bullet"
+ assert form_data["metric"]
+
+
+def test_bullet_native_form_data_round_trip_is_semantically_stable() -> None:
+ native = {
+ "viz_type": "bullet",
+ "datasource": "7__table",
+ "metric": {
+ "aggregate": "SUM",
+ "column": {"column_name": "Revenue"},
+ "expressionType": "SIMPLE",
+ "label": "Total Revenue",
+ },
+ "groupby": ["Region", "Team"],
+ "ranges": "100,250,500",
+ "range_labels": "Minimum,Target,Stretch",
+ "markers": "300",
+ "marker_labels": "Plan",
+ "marker_lines": "400",
+ "marker_line_labels": "Forecast",
+ "y_axis_format": "$,.0f",
+ "show_labels": False,
+ "show_legend": True,
+ "row_limit": 250,
+ "orderby": [["Region", True], ["Total Revenue", False]],
+ "adhoc_filters": [
+ {
+ "clause": "WHERE",
+ "expressionType": "SIMPLE",
+ "subject": "Status",
+ "operator": "==",
+ "comparator": "Active",
+ }
+ ],
+ }
+ config = BulletChartConfig.model_validate(native)
+ mapped = map_bullet_config(config)
+
+ assert mapped["metric"]["label"] == "Total Revenue"
+ assert mapped["groupby"] == ["Region", "Team"]
+ assert mapped["ranges"] == "100,250,500"
+ assert mapped["range_labels"] == "Minimum,Target,Stretch"
+ assert mapped["markers"] == "300"
+ assert mapped["marker_lines"] == "400"
+ assert mapped["show_labels"] is False
+ assert mapped["show_legend"] is True
+ assert mapped["orderby"][0] == ["Region", True]
+ assert mapped["orderby"][1][0]["label"] == "Total Revenue"
+ assert mapped["adhoc_filters"][0]["subject"] == "Status"
+
+
+def test_bullet_presentation_numbers_use_shortest_round_trip_safe_tokens() -> None:
+ ranges = [1.2345678901234567, 1.7976931348623157e308]
+ markers = [5e-324, -0.0]
+ marker_lines = [9.876543210987654e-200]
+ config = BulletChartConfig(
+ metric=_simple_metric(),
+ ranges=ranges,
+ markers=markers,
+ marker_lines=marker_lines,
+ show_legend=True,
+ )
+ mapped = map_bullet_config(config)
+
+ for key, expected in (
+ ("ranges", ranges),
+ ("markers", markers),
+ ("marker_lines", marker_lines),
+ ):
+ tokens = mapped[key].split(",")
+ assert [float(token) for token in tokens] == expected
+ assert all(
+ float(token).hex() == value.hex()
+ for token, value in zip(tokens, expected, strict=True)
+ )
+
+ round_trip = BulletChartConfig.model_validate(mapped)
+ assert round_trip.ranges == ranges
+ assert round_trip.markers == markers
+ assert round_trip.marker_lines == marker_lines
+
+ model = resolve_bullet_render_model(
+ [{"SUM(revenue)": 1.0}],
+ mapped,
+ )
+ assert model.ranges == ranges
+ assert model.markers == markers
+ assert model.marker_lines == marker_lines
+ assert (
+ "1.7976931348623157e+308"
+ in _generate_ascii_preview_from_data(
+ [{"SUM(revenue)": 1.0}], mapped
+ ).ascii_content
+ )
+ vega = _generate_vega_lite_preview_from_data([{"SUM(revenue)": 1.0}], mapped)
+ assert vega.specification["layer"]
+
+
+def test_bullet_native_saved_metric_and_legacy_metric_aliases() -> None:
+ saved = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": "saved_revenue"}
+ )
+ legacy = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": "sum__revenue"}
+ )
+ assert saved.metric.saved_metric is True
+ assert saved.metric.name == "saved_revenue"
+ assert legacy.metric.name == "sum__revenue"
+ assert legacy.metric.saved_metric is True
+
+
+@pytest.mark.parametrize("metric_name", ["sum__num", "sum__SP_POP_TOTL"])
+@pytest.mark.parametrize(
+ ("request_type", "request_fields"),
+ [
+ (GenerateChartRequest, {"dataset_id": 7}),
+ (UpdateChartRequest, {"identifier": 9}),
+ (UpdateChartPreviewRequest, {"dataset_id": 7}),
+ ],
+)
+def test_bullet_repository_metric_names_round_trip_as_saved_metrics_on_all_requests(
+ metric_name: str,
+ request_type: type[
+ GenerateChartRequest | UpdateChartRequest | UpdateChartPreviewRequest
+ ],
+ request_fields: dict[str, object],
+) -> None:
+ request = request_type.model_validate(
+ {**request_fields, "config": {"chart_type": "bullet", "metric": metric_name}}
+ )
+ config = request.config
+ assert isinstance(config, BulletChartConfig)
+ assert config.metric.saved_metric is True
+ assert map_bullet_config(config)["metric"] == metric_name
+
+
+@pytest.mark.parametrize(
+ ("native_operator", "typed_operator", "round_trip_operator"),
+ [
+ ("EQUALS", "=", "=="),
+ ("NOT_EQUALS", "!=", "!="),
+ ("LESS_THAN", "<", "<"),
+ ("LESS_THAN_OR_EQUAL", "<=", "<="),
+ ("GREATER_THAN", ">", ">"),
+ ("GREATER_THAN_OR_EQUAL", ">=", ">="),
+ ("IN", "IN", "IN"),
+ ("NOT_IN", "NOT IN", "NOT IN"),
+ ("LIKE", "LIKE", "LIKE"),
+ ("ILIKE", "ILIKE", "ILIKE"),
+ ("IS_NULL", "IS NULL", "IS NULL"),
+ ("IS_NOT_NULL", "IS NOT NULL", "IS NOT NULL"),
+ ],
+)
+def test_bullet_native_filter_operator_names_round_trip(
+ native_operator: str, typed_operator: str, round_trip_operator: str
+) -> None:
+ comparator: object
+ if native_operator.startswith("IS_"):
+ comparator = None
+ elif native_operator in {"IN", "NOT_IN"}:
+ comparator = ["North"]
+ else:
+ comparator = "North"
+ config = BulletChartConfig.model_validate(
+ {
+ "viz_type": "bullet",
+ "metric": "SavedRevenue",
+ "adhoc_filters": [
+ {
+ "clause": "WHERE",
+ "expressionType": "SIMPLE",
+ "subject": "Region",
+ "operator": native_operator,
+ "comparator": comparator,
+ }
+ ],
+ }
+ )
+
+ assert config.filters is not None
+ assert config.filters[0].op == typed_operator
+ mapped = map_bullet_config(config)
+ assert mapped["adhoc_filters"][0]["operator"] == round_trip_operator
+
+
+def test_bullet_legacy_label_only_saved_metric_adapter_is_strict_and_bounded() -> None:
+ config = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": {"label": "sum__num"}}
+ )
+ assert config.metric.saved_metric is True
+ assert map_bullet_config(config)["metric"] == "sum__num"
+
+ with pytest.raises(ValidationError):
+ BulletChartConfig.model_validate(
+ {
+ "viz_type": "bullet",
+ "metric": {"label": "sum__num", "aggregate": "SUM"},
+ }
+ )
+ with pytest.raises(ValidationError, match="at most 255"):
+ BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": {"label": "m" * 256}}
+ )
+
+
+@pytest.mark.parametrize(
+ "metric",
+ [
+ "SavedRevenue",
+ {
+ "aggregate": "SUM",
+ "column": {"column_name": "Revenue"},
+ "expressionType": "SIMPLE",
+ "label": "Simple Revenue",
+ },
+ {
+ "aggregate": None,
+ "column": None,
+ "expressionType": "SQL",
+ "sqlExpression": "SUM(Revenue)",
+ "label": "SQL Revenue",
+ },
+ ],
+)
+def test_bullet_all_metric_shapes_round_trip_full_native_presentation(
+ metric: object,
+) -> None:
+ native = {
+ "viz_type": "bullet",
+ "metric": metric,
+ "groupby": ["Region"],
+ "ranges": "50,100",
+ "range_labels": "Low,High",
+ "markers": "75",
+ "marker_labels": "Plan",
+ "marker_lines": "90",
+ "marker_line_labels": "Forecast",
+ "y_axis_format": "$,.0f",
+ "show_labels": True,
+ "show_legend": True,
+ }
+ mapped = map_bullet_config(BulletChartConfig.model_validate(native))
+ assert validate_merged_bullet_form_data(mapped) is not None
+ assert mapped["groupby"] == ["Region"]
+ assert mapped["ranges"] == "50,100"
+ assert mapped["marker_line_labels"] == "Forecast"
+
+
+def test_bullet_rejects_invalid_roles_and_output_collisions() -> None:
+ with pytest.raises(ValidationError, match="physical dimension"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[{"name": "region", "aggregate": "COUNT"}],
+ )
+ with pytest.raises(ValidationError, match="Duplicate Bullet dimension"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[{"name": "Region"}, {"name": "Region"}],
+ )
+ with pytest.raises(ValidationError, match="conflicts with a dimension"):
+ BulletChartConfig(
+ metric={"name": "revenue", "aggregate": "SUM", "label": "Region"},
+ dimensions=[{"name": "Region", "label": "Friendly"}],
+ )
+
+
+def test_bullet_accepts_short_labels_and_rejects_bad_order_target() -> None:
+ config = BulletChartConfig(
+ metric=_simple_metric(), ranges=[1, 2], range_labels=["Only one"]
+ )
+ assert config.range_labels == ["Only one"]
+ with pytest.raises(ValidationError, match="unknown: not_a_role"):
+ BulletChartConfig(
+ metric=_simple_metric(), dimensions=[], order_by=[{"column": "not_a_role"}]
+ )
+
+
+def test_bullet_dimension_labels_are_input_aliases_not_result_aliases() -> None:
+ config = BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Total"},
+ dimensions=[
+ {"name": "Team", "label": "Region"},
+ {"name": "Region", "label": "Market"},
+ ],
+ # The exact physical Region must win over Team's display label.
+ order_by=[
+ {"column": "Region", "ascending": True},
+ {"column": "Revenue", "ascending": False},
+ ],
+ )
+ form_data = map_bullet_config(config)
+ assert form_data["groupby"] == ["Team", "Region"]
+ assert form_data["orderby"] == [
+ ["Region", True],
+ [form_data["metric"], False],
+ ]
+
+ label_order = map_bullet_config(
+ BulletChartConfig(
+ metric=config.metric,
+ dimensions=config.dimensions,
+ order_by=[{"column": "Market"}],
+ )
+ )
+ assert label_order["orderby"] == [["Region", False]]
+
+ model = resolve_bullet_render_model(
+ [{"Team": "Blue", "Region": "North", "Total": 10}], form_data
+ )
+ assert model.dimensions == ["Team", "Region"]
+ assert [model.rows[0][name] for name in model.dimensions] == ["Blue", "North"]
+
+
+def test_bullet_rejects_ambiguous_display_alias_for_ordering() -> None:
+ with pytest.raises(ValidationError, match="ambiguous display alias"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[
+ {"name": "Region", "label": "Area"},
+ {"name": "Team", "label": "area"},
+ ],
+ order_by=[{"column": "AREA"}],
+ )
+
+
+@pytest.mark.parametrize(
+ ("metric", "order_target", "output"),
+ [
+ (
+ {"name": "SavedRevenue", "saved_metric": True, "label": "Friendly"},
+ "Friendly",
+ "SavedRevenue",
+ ),
+ (
+ {"name": "Revenue", "aggregate": "SUM", "label": "Simple Total"},
+ "Revenue",
+ "Simple Total",
+ ),
+ (
+ {"sql_expression": "SUM(Revenue)", "label": "SQL Total"},
+ "SQL Total",
+ "SQL Total",
+ ),
+ ],
+)
+def test_bullet_metric_shapes_share_physical_dimension_output_contract(
+ metric: dict[str, object], order_target: str, output: str
+) -> None:
+ config = BulletChartConfig(
+ metric=metric,
+ dimensions=[{"name": "Region", "label": "Market"}],
+ order_by=[{"column": order_target}],
+ )
+ form_data = map_bullet_config(config)
+ assert form_data["groupby"] == ["Region"]
+ assert form_data["orderby"] == [[form_data["metric"], False]]
+ model = resolve_bullet_render_model([{"Region": "North", output: 12}], form_data)
+ assert model.metric_field == output
+ assert model.dimensions == ["Region"]
+
+
+def test_bullet_mapper_preserves_omission_and_honors_explicit_values() -> None:
+ omitted = map_bullet_config(BulletChartConfig(metric=_simple_metric()))
+ explicit = map_bullet_config(
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[],
+ filters=[],
+ ranges=[],
+ show_labels=False,
+ show_legend=False,
+ row_limit=42,
+ time_range=None,
+ )
+ )
+ for key in (
+ "groupby",
+ "adhoc_filters",
+ "ranges",
+ "show_labels",
+ "show_legend",
+ "row_limit",
+ "time_range",
+ ):
+ assert key not in omitted
+ assert explicit["groupby"] == []
+ assert explicit["adhoc_filters"] == []
+ assert explicit["ranges"] == ""
+ assert explicit["show_labels"] is False
+ assert explicit["show_legend"] is False
+ assert explicit["row_limit"] == 42
+ assert explicit["time_range"] is None
+
+
+def test_bullet_registry_schema_and_recommendation_metadata() -> None:
+ from superset.mcp_service.app import get_default_instructions
+ from superset.mcp_service.chart.registry import display_name_for_viz_type, get
+
+ plugin = get("bullet")
+ assert plugin is not None
+ assert plugin.resolve_viz_type(None) == "bullet"
+ assert display_name_for_viz_type("bullet") == "Bullet Chart"
+ assert "bullet" in VALID_CHART_TYPES
+ discovered = _get_chart_type_schema_impl("bullet")
+ assert discovered["chart_type"] == "bullet"
+ assert discovered["examples"][0]["ranges"] == [100000, 250000, 500000]
+ assert _VIZ_CATEGORY["bullet"] == "bullet"
+ candidates = _candidates_single_numeric(
+ DataColumn(
+ name="Revenue",
+ display_name="Revenue",
+ data_type="numeric",
+ sample_values=[1],
+ null_count=0,
+ unique_count=1,
+ ),
+ row_count=1,
+ )
+ assert "bullet chart" in candidates
+ guidance = get_default_instructions()
+ assert 'chart_type="bullet": Bullet Chart' in guidance
+ assert "waterfall, gantt, bullet, bubble_v2, and interactive_pivot" in guidance
+
+
+def test_bullet_dataset_normalization_canonicalizes_every_reference() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "Region", "type": "VARCHAR"},
+ {"name": "OrderDate", "type": "TIMESTAMP", "is_temporal": True},
+ {"name": "Status", "type": "VARCHAR"},
+ ],
+ available_metrics=[],
+ )
+ config = BulletChartConfig(
+ metric={"name": "revenue", "aggregate": "SUM"},
+ dimensions=[{"name": "region"}],
+ temporal_column="orderdate",
+ filters=[{"column": "status", "op": "=", "value": "active"}],
+ order_by=[{"column": "region", "ascending": True}],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ normalized = plugin.normalize_column_refs(config, context)
+ assert normalized.metric.name == "Revenue"
+ assert normalized.dimensions[0].name == "Region"
+ assert normalized.temporal_column == "OrderDate"
+ assert normalized.filters[0].column == "Status"
+ assert normalized.order_by[0].column == "Region"
+ assert normalized.model_fields_set == config.model_fields_set
+
+
+def test_bullet_dataset_normalization_rejects_ambiguous_casefold_candidates() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "revenue", "type": "NUMERIC", "is_numeric": True},
+ ],
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ config = BulletChartConfig(metric={"name": "REVENUE", "aggregate": "SUM"})
+ with pytest.raises(ValueError, match="Revenue, revenue"):
+ plugin.normalize_column_refs(config, context)
+
+
+def test_bullet_numeric_output_constraint_rejects_text_min() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[{"name": "status", "type": "VARCHAR"}],
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ config = BulletChartConfig(metric={"name": "status", "aggregate": "MIN"})
+ with patch.object(DatasetValidator, "_get_dataset_context", return_value=context):
+ error = plugin.post_map_validate(config, {}, dataset_id=7)
+ assert error is not None
+ assert error.error_type == "non_numeric_bullet_metric"
+
+
+@pytest.mark.parametrize("reverse_metadata", [False, True])
+def test_bullet_exact_case_type_and_role_resolution_is_order_independent(
+ reverse_metadata: bool,
+) -> None:
+ from superset.mcp_service.chart.registry import get
+
+ columns = [
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "revenue", "type": "VARCHAR", "is_numeric": False},
+ {"name": "Region", "type": "VARCHAR"},
+ ]
+ if reverse_metadata:
+ columns.reverse()
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=columns,
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+
+ numeric = BulletChartConfig(metric={"name": "Revenue", "aggregate": "SUM"})
+ text = BulletChartConfig(metric={"name": "revenue", "aggregate": "MIN"})
+ with patch.object(DatasetValidator, "_get_dataset_context", return_value=context):
+ assert plugin.post_map_validate(numeric, {}, dataset_id=7) is None
+ error = plugin.post_map_validate(text, {}, dataset_id=7)
+ assert error is not None
+ assert error.error_type == "non_numeric_bullet_metric"
+
+ roles = BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM"},
+ dimensions=[{"name": "revenue"}, {"name": "Region"}],
+ filters=[{"column": "revenue", "op": "=", "value": "retail"}],
+ order_by=[{"column": "revenue", "ascending": True}],
+ )
+ normalized = plugin.normalize_column_refs(roles, context)
+ assert normalized.metric.name == "Revenue"
+ assert [dimension.name for dimension in normalized.dimensions or []] == [
+ "revenue",
+ "Region",
+ ]
+ assert normalized.filters
+ assert normalized.filters[0].column == "revenue"
+ assert normalized.order_by[0].column == "revenue"
+
+ ambiguous = BulletChartConfig(metric={"name": "REVENUE", "aggregate": "SUM"})
+ with pytest.raises(
+ AmbiguousDatasetReferenceError, match="Bullet metric column reference"
+ ):
+ plugin.normalize_column_refs(ambiguous, context)
+
+
+@pytest.mark.parametrize("reverse_metadata", [False, True])
+def test_generic_aggregation_validation_uses_exact_case_before_type(
+ reverse_metadata: bool,
+) -> None:
+ from superset.mcp_service.chart.schemas import PieChartConfig
+
+ columns = [
+ {"name": "Amount", "type": "BIGINT", "is_numeric": True},
+ {"name": "amount", "type": "VARCHAR", "is_numeric": False},
+ ]
+ if reverse_metadata:
+ columns.reverse()
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=columns,
+ available_metrics=[],
+ )
+
+ assert (
+ DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "Amount", "aggregate": "SUM"}).metric],
+ context,
+ )
+ == []
+ )
+ errors = DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "amount", "aggregate": "SUM"}).metric],
+ context,
+ )
+ assert errors
+ assert errors[0].error_type == "invalid_aggregation"
+
+ ambiguous = DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "AMOUNT", "aggregate": "SUM"}).metric],
+ context,
+ )
+ assert ambiguous
+ assert ambiguous[0].error_type == "ambiguous_column_reference"
+
+ valid, error = DatasetValidator.validate_against_dataset(
+ PieChartConfig(
+ dimension={"name": "amount"},
+ metric={"name": "Amount", "aggregate": "SUM"},
+ ),
+ 7,
+ dataset_context=context,
+ )
+ assert valid is True
+ assert error is None
+
+
+@pytest.mark.parametrize(
+ ("metric", "field"),
+ [
+ ({"name": "Revenue", "aggregate": "SUM", "label": "Simple"}, "Simple"),
+ ({"name": "SavedRevenue", "saved_metric": True}, "SavedRevenue"),
+ ({"sql_expression": "SUM(Revenue)", "label": "SQL Total"}, "SQL Total"),
+ ],
+)
+def test_bullet_result_roles_are_exact_for_every_metric_shape(
+ metric: dict[str, object], field: str
+) -> None:
+ form_data = map_bullet_config(BulletChartConfig(metric=metric))
+ model = resolve_bullet_render_model([{field.swapcase(): "12.5"}], form_data)
+ assert model.metric_field == field.swapcase()
+ assert model.measures == [12.5]
+
+
+@pytest.mark.parametrize(
+ "rows, message",
+ [
+ ([{"other": 123}], "missing"),
+ ([{"Revenue": "not a number"}], "non-numeric text"),
+ ([{"Revenue": math.nan}], "NaN or infinite"),
+ ([{"Revenue": math.inf}], "NaN or infinite"),
+ ([{"Revenue": 1}, {}], "row 1.*missing"),
+ ([{"REVENUE": 1, "revenue": 2}], "ambiguous"),
+ ],
+)
+def test_bullet_result_validation_rejects_malformed_rows(
+ rows: list[dict[str, object]], message: str
+) -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "amount", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ with pytest.raises(BulletOutputError, match=message):
+ resolve_bullet_render_model(rows, form_data)
+
+
+def test_bullet_result_validation_accepts_null_and_numeric_strings() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "amount", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Region"}],
+ )
+ )
+ model = resolve_bullet_render_model(
+ [
+ {"Region": "North", "Revenue": None},
+ {"Region": "South", "Revenue": " 4.25 "},
+ ],
+ form_data,
+ )
+ assert model.measures == [0.0, 4.25]
+
+
+@pytest.mark.parametrize(
+ ("presentation", "message"),
+ [
+ ({"ranges": "10,nope"}, r"ranges\[1\].*not numeric"),
+ ({"markers": "NaN"}, r"markers\[0\].*NaN or infinite"),
+ ],
+)
+def test_bullet_result_validation_rejects_malformed_presentation(
+ presentation: dict[str, object], message: str
+) -> None:
+ form_data = {
+ **map_bullet_config(
+ BulletChartConfig(metric={"name": "amount", "aggregate": "SUM"})
+ ),
+ **presentation,
+ }
+ with pytest.raises(BulletOutputError, match=message):
+ resolve_bullet_render_model([{"SUM(amount)": 1}], form_data)
+
+
+def test_bullet_compile_accepts_empty_ungrouped_result() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {"queries": [{"data": []}]}
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ assert result.success is True
+ assert result.row_count == 0
+
+
+def test_bullet_compile_inspects_top_level_and_query_error_envelopes() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(metric={"name": "Revenue", "aggregate": "SUM"})
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {
+ "status": "success",
+ "queries": [{"status": "failed", "message": "warehouse timeout"}],
+ }
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ assert result.success is False
+ assert "warehouse timeout" in (result.error or "")
+
+
+_MALFORMED_QUERY_ENVELOPES: list[object] = [
+ None,
+ [],
+ {},
+ {"queries": None},
+ {"queries": []},
+ {"queries": [None]},
+ {"queries": [{}]},
+ {"queries": [{"data": None}]},
+ {"queries": [{"data": []}, {"data": "not-an-array"}]},
+ {
+ "queries": [
+ {
+ "data": [{"Revenue": 12}],
+ "colnames": ["Revenue"],
+ "coltypes": [],
+ }
+ ]
+ },
+]
+
+
+def _compile_bullet_with_result(result: object) -> Any:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = result
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ return _compile_chart(form_data, 7)
+
+
+@pytest.mark.parametrize("envelope", _MALFORMED_QUERY_ENVELOPES)
+def test_bullet_compile_returns_stable_error_for_malformed_envelopes(
+ envelope: object,
+) -> None:
+ result = _compile_bullet_with_result(envelope)
+ assert result.success is False
+ assert result.error_code == "CHART_COMPILE_FAILED"
+ assert result.error_obj is not None
+ assert result.error_obj.error_type == "compile_error"
+
+
+@pytest.mark.parametrize(
+ ("data", "expected_code", "expected_type"),
+ [
+ ([1], "CHART_COMPILE_FAILED", "compile_error"),
+ (
+ [{"Revenue": 10**10000}],
+ "CHART_COMPILE_FAILED",
+ "compile_error",
+ ),
+ ],
+)
+def test_bullet_compile_returns_malformed_output_for_bad_rows(
+ data: list[object],
+ expected_code: str,
+ expected_type: str,
+) -> None:
+ result = _compile_bullet_with_result({"queries": [{"data": data}]})
+ assert result.success is False
+ assert result.error_code == expected_code
+ assert result.error_obj is not None
+ assert result.error_obj.error_type == expected_type
+
+
+def test_bullet_shared_query_builder_matches_frontend_build_query() -> None:
Review Comment:
Fixed in fa297506b5c3dbbbc383135c5c388ac7c018af2c: renamed the Python test to test_bullet_query_builder_forwards_configured_roles_sort_and_limit, which accurately describes its backend-only scope. Added a separate frontend buildQuery test asserting groupby, metric, explicit ordering, and row limit, so dropping those controls on the frontend fails a frontend test rather than relying on a claimed cross-language parity check. Both Bullet frontend suites passed (19 tests).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4172337393
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1857 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
return None
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
Review Comment:
fixed in df03f8e50b: bytes/bytearray/memoryview values go through the binary-column contract (UTF-8 text or base64: prefix) with size bounds.
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,427 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ matches = [
+ item
+ for item in saved_metrics
+ if item == name or item.casefold() == name.casefold()
+ ]
+ if len(set(matches)) != 1:
Review Comment:
fixed in df03f8e50b: an exact saved-metric name wins; only case-folded lookups must be unambiguous.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-6009989701
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3906289002
##########
superset/mcp_service/chart/tool/get_chart_preview.py:
##########
@@ -700,6 +701,126 @@ def _bar_chart_spec(
},
}
+ def _bullet_chart_spec(
+ self, fields: List[str], field_types: Dict[str, str] | None = None
+ ) -> Dict[str, Any]:
+ """Create a horizontal Bullet spec from saved native form_data."""
+ from superset.mcp_service.chart.preview_utils import (
+ _bullet_numeric_tokens,
+ _form_column_label,
+ _form_metric_label,
+ )
+ from superset.utils import json as utils_json
+
+ field_types = field_types or {}
+ form_data = self._get_form_data() or {}
+
+ def result_field(label: str | None) -> str | None:
+ if label in fields:
+ return label
+ if label:
+ matches = [
+ field for field in fields if field.casefold() == label.casefold()
+ ]
+ if len(matches) == 1:
+ return matches[0]
+ return None
+
+ metric_field = result_field(_form_metric_label(form_data.get("metric")))
+ if metric_field is None:
+ metric_field = next(
+ (
+ field
+ for field in reversed(fields)
+ if field_types.get(field) == "quantitative"
+ ),
+ fields[-1] if fields else "metric",
+ )
+ dimensions = [
+ field
+ for column in form_data.get("groupby") or []
+ if (field := result_field(_form_column_label(column)))
+ ]
+ if not dimensions:
+ dimensions = [field for field in fields if field != metric_field]
+
+ category_field = "__mcp_bullet_category"
+ calculate = (
+ " + ', ' + ".join(
+ f"toString(datum[{utils_json.dumps(field)}])" for field in dimensions
+ )
+ if dimensions
+ else "'Measure'"
+ )
+ y_encoding = {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(dimensions) if dimensions else None,
+ "sort": None,
+ }
+ layers: list[dict[str, Any]] = []
+ for index, threshold in enumerate(
+ sorted(_bullet_numeric_tokens(form_data.get("ranges")), reverse=True)
+ ):
+ layers.append(
+ {
+ "mark": {
+ "type": "rect",
+ "opacity": max(0.08, 0.28 - index * 0.04),
+ },
+ "encoding": {
+ "x": {"datum": 0, "type": "quantitative"},
+ "x2": {"datum": threshold},
+ },
+ }
+ )
Review Comment:
**Suggestion:** Range rectangles have no category `y` encoding, so grouped Bullet previews render each threshold as one chart-wide band instead of a background behind every bullet row. [logic error]
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Sometimes`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=644c73c7093f4c519955df442e608dab&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=644c73c7093f4c519955df442e608dab&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/mcp_service/chart/tool/get_chart_preview.py
**Line:** 762:776
**Comment:**
*Logic Error: Range rectangles have no category `y` encoding, so grouped Bullet previews render each threshold as one chart-wide band instead of a background behind every bullet row.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=7cd859131e86d2bc9e5997fb1037c11e9d85254a9b1199b0e8e128b01ca2be52&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=7cd859131e86d2bc9e5997fb1037c11e9d85254a9b1199b0e8e128b01ca2be52&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5500503692
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3927155012
##########
superset/mcp_service/chart/tool/get_chart_preview.py:
##########
@@ -700,6 +701,126 @@ def _bar_chart_spec(
},
}
+ def _bullet_chart_spec(
+ self, fields: List[str], field_types: Dict[str, str] | None = None
+ ) -> Dict[str, Any]:
+ """Create a horizontal Bullet spec from saved native form_data."""
+ from superset.mcp_service.chart.preview_utils import (
+ _bullet_numeric_tokens,
+ _form_column_label,
+ _form_metric_label,
+ )
+ from superset.utils import json as utils_json
+
+ field_types = field_types or {}
+ form_data = self._get_form_data() or {}
+
+ def result_field(label: str | None) -> str | None:
+ if label in fields:
+ return label
+ if label:
+ matches = [
+ field for field in fields if field.casefold() == label.casefold()
+ ]
+ if len(matches) == 1:
+ return matches[0]
+ return None
+
+ metric_field = result_field(_form_metric_label(form_data.get("metric")))
+ if metric_field is None:
+ metric_field = next(
+ (
+ field
+ for field in reversed(fields)
+ if field_types.get(field) == "quantitative"
+ ),
+ fields[-1] if fields else "metric",
+ )
+ dimensions = [
+ field
+ for column in form_data.get("groupby") or []
+ if (field := result_field(_form_column_label(column)))
+ ]
+ if not dimensions:
+ dimensions = [field for field in fields if field != metric_field]
+
+ category_field = "__mcp_bullet_category"
+ calculate = (
+ " + ', ' + ".join(
+ f"toString(datum[{utils_json.dumps(field)}])" for field in dimensions
+ )
+ if dimensions
+ else "'Measure'"
+ )
+ y_encoding = {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(dimensions) if dimensions else None,
+ "sort": None,
+ }
+ layers: list[dict[str, Any]] = []
+ for index, threshold in enumerate(
+ sorted(_bullet_numeric_tokens(form_data.get("ranges")), reverse=True)
+ ):
+ layers.append(
+ {
+ "mark": {
+ "type": "rect",
+ "opacity": max(0.08, 0.28 - index * 0.04),
+ },
+ "encoding": {
+ "x": {"datum": 0, "type": "quantitative"},
+ "x2": {"datum": threshold},
+ },
+ }
+ )
Review Comment:
Fixed in 703bb6cd. The divergent saved-chart `_bullet_chart_spec` was removed, so every saved/unsaved/cache Bullet Vega path now uses `_generate_bullet_vega_lite_preview`; each range rect (and optional range label) carries the same collision-safe category `y` encoding as the bar. Covered by grouped saved-preview and public FastMCP Vega assertions; the affected focused suite passed (1,451 tests).
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -323,6 +325,102 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if isinstance(metric, str):
+ return metric
+ if not isinstance(metric, dict):
+ return None
+ if label := metric.get("label"):
+ return label if isinstance(label, str) else None
+ column = metric.get("column")
+ column_name = column.get("column_name") if isinstance(column, dict) else column
+ aggregate = metric.get("aggregate")
+ if isinstance(column_name, str) and isinstance(aggregate, str):
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if isinstance(column, str):
+ return column
+ if not isinstance(column, dict):
+ return None
+ for key in ("label", "column_name"):
+ if isinstance(value := column.get(key), str) and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve a query result field by exact match, then unambiguous casefold."""
+ if label is None:
+ return None
+ if label in row:
+ return label
+ matches = [field for field in row if field.casefold() == label.casefold()]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _bullet_result_roles(
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> tuple[str | None, list[str]]:
+ """Resolve Bullet metric and full category hierarchy in query output."""
+ if not data:
+ return None, []
+ first_row = data[0]
+ metric_field = _canonical_result_field(
+ _form_metric_label(form_data.get("metric")), first_row
+ )
+ if metric_field is None:
+ metric_field = next(
+ (
+ field
+ for field, value in first_row.items()
+ if isinstance(value, (int, float)) and not _is_nan(value)
+ ),
+ None,
+ )
Review Comment:
Already fixed at the reviewed head and retained in 703bb6cd. `resolve_bullet_render_model` calls `_require_result_field` for the declared metric alias and fails missing/ambiguous aliases; it never selects a first numeric field. `test_bullet_result_validation_rejects_malformed_rows` covers a numeric unrelated field (`other: 123`) and exact/casefold ambiguity, and the full MCP suite passed (4,386 tests).
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -469,11 +567,126 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _generate_bullet_vega_lite_preview(
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview | None:
+ """Build a horizontal layered preview faithful to Bullet result roles."""
+ metric_field, dimensions = _bullet_result_roles(data, form_data)
+ if metric_field is None:
+ return None
+
+ category_field = "__mcp_bullet_category"
+ values = []
+ for index, row in enumerate(data):
+ copied = dict(row)
+ copied[category_field] = (
+ ", ".join(str(row.get(field, "")) for field in dimensions)
+ if dimensions
+ else str(index + 1)
+ )
+ values.append(copied)
+
+ y_encoding = {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(dimensions) if dimensions else None,
+ "sort": None,
+ }
+ tooltip = [
+ *({"field": field, "type": "nominal"} for field in dimensions),
+ {"field": metric_field, "type": "quantitative"},
+ ]
+ layers: list[dict[str, Any]] = []
+ for index, threshold in enumerate(
+ sorted(_bullet_numeric_tokens(form_data.get("ranges")), reverse=True)
+ ):
+ layers.append(
+ {
+ "mark": {
+ "type": "rect",
+ "opacity": max(0.08, 0.28 - index * 0.04),
+ },
+ "encoding": {
+ "x": {"datum": 0, "type": "quantitative"},
+ "x2": {"datum": threshold},
Review Comment:
Fixed in 703bb6cd. The shared strict Bullet renderer assigns `"y": y_encoding` to every range rect and range-label layer, and the old duplicate saved renderer was removed. Direct saved-preview and public FastMCP grouped tests assert every rect uses the bar category encoding.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2014,6 +2016,385 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Typed contract for the ECharts Bullet visualization (viz_type ``bullet``).
+
+ Semantic field names are exposed to MCP clients while validation aliases and
+ the native adapter accept saved Explore ``form_data`` without weakening the
+ unknown-field checks that catch misspelled controls.
+ """
+
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description=(
+ "Numeric measure shown by each bullet bar. Use aggregate for a SIMPLE "
+ "metric, saved_metric=True for a dataset metric, or sql_expression "
+ "with a unique label."
+ ),
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Optional category hierarchy; the frontend renders one bullet row per "
+ "unique combination (native form_data: groupby). Omit to preserve a "
+ "saved hierarchy on update; pass [] to clear it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(
+ None,
+ description=(
+ "Structured WHERE filters. Native SIMPLE adhoc_filters are accepted; "
+ "free-form SQL filters are rejected."
+ ),
+ max_length=100,
+ )
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Optional Superset time range such as 'Last 30 days' or "
+ "'2025-01-01 : 2025-12-31'. Set temporal_column to choose its column."
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum grouped bullet rows returned by the query",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description=(
+ "Stable row ordering by a dimension name or by the metric's output "
+ "label/name. Native orderby pairs and order_by_cols JSON pairs are "
+ "accepted for saved-form-data round trips."
+ ),
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ legacy = re.fullmatch(
+ r"(sum|avg|min|max|count|count_distinct)__(.+)",
+ value,
+ flags=re.IGNORECASE,
+ )
+ if legacy:
+ return {
+ "name": legacy.group(2),
+ "aggregate": legacy.group(1).upper(),
+ }
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict) or "expressionType" not in value:
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ data.setdefault("temporal_column", subject)
+ if isinstance(comparator, str) and comparator.casefold() != "no filter":
+ data.setdefault("time_range", comparator)
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {"==": "=", "IS_NOT_NULL": "IS NOT NULL"}
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ if data.get("viz_type") == "bullet":
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ ):
+ data.pop(key, None)
+
+ marker_key = "_mcp_dashboard_time_filter_subject"
+ if marker := data.pop(marker_key, None):
+ if not isinstance(marker, str):
+ raise ValueError(f"{marker_key} must be a physical column name")
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
+ {"name": item} if isinstance(item, str) else item
+ for item in data[key]
Review Comment:
Addressed and verified at 703bb6cd. The pre-validator canonicalizes both aliases and rejects unequal lists before alias precedence can apply; equivalent lists are order-independent. A new regression also documents why schema-only identity remains exact-spelling (`Region` and `region` may be distinct quoted columns), while dataset-aware resolution remains exact-first and rejects ambiguous casefold matches.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1065,6 +1066,155 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ return ",".join(
+ format(value, "g") if isinstance(value, float) else value for value in values
+ )
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ metric_output = metric if isinstance(metric, str) else metric.get("label")
+ metric_targets = {
+ candidate.casefold()
+ for candidate in (
+ config.metric.name,
+ config.metric.label,
+ metric_output,
+ )
+ if candidate
+ }
+ dimensions = config.dimensions or []
+ dimension_targets = {
+ candidate.casefold(): dimension.name
+ for dimension in dimensions
+ for candidate in (dimension.name, dimension.label)
+ if candidate and dimension.name
+ }
+ form_data["orderby"] = [
+ [
+ (
+ metric
+ if order.column.casefold() in metric_targets
+ else dimension_targets[order.column.casefold()]
+ ),
+ order.ascending,
+ ]
+ for order in config.order_by
+ ]
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
Review Comment:
Already fixed in the production shared merge path and retained in 703bb6cd. `merge_update_form_data` treats `time_range` as authoritative, copies the saved provenance-owned binding, replaces its comparator in place, and retains the saved subject when `temporal_column` is omitted. `test_bullet_time_range_only_updates_saved_subject_on_every_update_path` covers immediate, preview-first, and cached paths for active and cleared ranges. The unused compatibility wrapper was removed so no tests exercise dead logic.
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,272 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import BulletChartConfig, ColumnRef
+from superset.mcp_service.chart.validation.dataset_validator import (
+ DatasetValidator,
+ is_numeric_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ f"Ambiguous Bullet {role} {name!r}; exact-case matches are: "
+ f"{', '.join(matches)}"
+ )
+ return matches[0] if matches else name
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate(
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=f"Bullet {role} {name!r} is ambiguous by case",
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ column = next(
+ (
+ column
+ for column in dataset_context.available_columns
+ if column["name"].casefold() == (metric.name or "").casefold()
+ ),
+ None,
+ )
+ if column is None or is_numeric_column(column):
+ return None
Review Comment:
Already fixed at the reviewed head and retained in 703bb6cd. Dataset resolution is exact-name first; only a unique casefold candidate is accepted. `test_bullet_exact_case_type_and_role_resolution_is_order_independent` proves `Revenue` (numeric) and `revenue` (text) select their own types regardless of metadata order, while `REVENUE` is rejected as ambiguous.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1065,6 +1066,155 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ return ",".join(
+ format(value, "g") if isinstance(value, float) else value for value in values
Review Comment:
Already fixed at the reviewed head and retained in 703bb6cd. Presentation numbers use shortest round-trip-safe finite spelling rather than `format(..., "g")`; `test_bullet_presentation_numbers_use_shortest_round_trip_safe_tokens` covers `123456.789`, binary64 extremes, and exact semantic round trips.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1065,6 +1121,660 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+ """
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+ merge_bullet_form_data(existing_form_data, new_form_data)
+
+
+def _currency_form_value(value: CurrencyFormat | None) -> dict[str, str] | None:
+ """Return the native value for an explicitly supplied currency control."""
+ return value.to_form_data() if value is not None else None
+
+
+def _column_names(value: Sequence[ColumnRef] | None) -> list[str | None] | None:
+ """Return a native column-name list while retaining an explicit null."""
+ return [column.name for column in value] if value is not None else None
+
+
+def _table_sort_value(value: Sequence[str | SortByConfig] | None) -> list[str] | None:
+ """Return the native Table sort control for an explicit typed value."""
+ if value is None:
+ return None
+ return [
+ json.dumps(
+ [entry.column, entry.ascending]
+ if isinstance(entry, SortByConfig)
+ else [entry, False]
+ )
+ for entry in value
+ ]
+
+
+def _table_column_config_value(value: Any) -> dict[str, Any] | None:
+ """Return Table column config without losing an explicit null or empty map."""
+ if value is None:
+ return None
+ return {
+ label: column.model_dump(by_alias=True, exclude_unset=True)
+ for label, column in value.items()
+ }
+
+
+# Mappers intentionally omit optional controls so fresh charts use the frontend
+# defaults. During a same-viz update, however, an explicitly supplied false,
+# null, or empty value must block preservation of the saved native key. Keep the
+# typed-to-native relationship declarative so every update path shares it.
+_FormValueConverter = Callable[[Any], Any]
+_FormControlMap = dict[str, tuple[str, _FormValueConverter]]
+
+_COMMON_EXPLICIT_FORM_CONTROLS: _FormControlMap = {
+ "color_scheme": ("color_scheme", lambda value: value),
+ "currency_format": ("currency_format", _currency_form_value),
+ "show_value": ("show_value", lambda value: value),
+}
+
+_CHART_EXPLICIT_FORM_CONTROLS: dict[str, _FormControlMap] = {
+ "table": {
+ "sort_by": ("order_by_cols", _table_sort_value),
+ "column_config": ("column_config", _table_column_config_value),
+ },
+ "xy": {
+ "group_by": ("groupby", _column_names),
+ "series_limit": ("series_limit", lambda value: value),
+ "stacked": ("stack", lambda value: "Stack" if value else None),
+ "orientation": ("orientation", lambda value: value),
+ "legend_orientation": ("legendOrientation", lambda value: value),
+ "x_axis_time_format": ("x_axis_time_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "mixed_timeseries": {
+ "group_by": ("groupby", _column_names),
+ "group_by_secondary": ("groupby_b", _column_names),
+ "currency_format_secondary": (
+ "currency_format_secondary",
+ _currency_form_value,
+ ),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "waterfall": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "big_number": {
+ "subheader": ("subheader", lambda value: value),
+ "y_axis_format": ("y_axis_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "compare_lag": ("compare_lag", lambda value: value),
+ "time_format": ("time_format", lambda value: value),
+ "aggregation": ("aggregation", lambda value: value),
+ },
+ "handlebars": {
+ "style_template": ("styleTemplate", lambda value: value),
+ "columns": ("all_columns", _column_names),
+ "groupby": ("groupby", _column_names),
+ "metrics": ("metrics", _column_names),
+ },
+ "pivot_table": {
+ "date_format": ("date_format", lambda value: value),
+ },
+ "interactive_pivot": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "series_limit": ("series_limit", lambda value: value),
+ "date_format": ("date_format", lambda value: value),
+ "column_sort": ("colOrder", lambda value: value),
+ },
+}
+
+
+def _apply_explicit_form_controls( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply typed controls whose mapper omission represents a native clear."""
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ controls = {
+ **_COMMON_EXPLICIT_FORM_CONTROLS,
+ **_CHART_EXPLICIT_FORM_CONTROLS.get(config.chart_type, {}),
+ }
+ for field_name, (native_key, convert) in controls.items():
+ if field_name in explicit_fields:
+ converted = convert(getattr(config, field_name))
+ is_clear = (
+ converted is None
+ or converted is False
+ or converted == ""
+ or converted in ([], {})
+ )
+ if is_clear:
+ new_form_data[native_key] = converted
+
+ axis_controls = {
+ "xy": (
+ ("x_axis", "x_axis_title", "x_axis_format", None),
+ ("y_axis", "y_axis_title", "y_axis_format", "y_axis_scale"),
+ ),
+ "mixed_timeseries": (
+ ("x_axis", "xAxisTitle", "x_axis_time_format", None),
+ ("y_axis", "yAxisTitle", "y_axis_format", "logAxis"),
+ (
+ "y_axis_secondary",
+ "yAxisTitleSecondary",
+ "y_axis_format_secondary",
+ "logAxisSecondary",
+ ),
+ ),
+ }
+ if config.chart_type in axis_controls:
+ for field_name, title_key, format_key, scale_key in axis_controls[
+ config.chart_type
+ ]:
+ if field_name not in explicit_fields:
+ continue
+ axis = getattr(config, field_name)
+ if axis is None:
+ new_form_data[title_key] = None
+ new_form_data[format_key] = None
+ if scale_key:
+ new_form_data[scale_key] = None
+ continue
+ axis_fields = set(axis.model_fields_set)
+ if "title" in axis_fields:
+ new_form_data[title_key] = axis.title
+ if "format" in axis_fields:
+ new_form_data[format_key] = axis.format
+ if scale_key and "scale" in axis_fields:
+ new_form_data[scale_key] = axis.scale
+
+ if config.chart_type == "xy" and "legend" in explicit_fields:
+ legend = config.legend
+ if legend is None:
+ new_form_data["show_legend"] = None
+ new_form_data["legendOrientation"] = None
+ else:
+ legend_fields = set(legend.model_fields_set)
+ if "show" in legend_fields:
+ new_form_data["show_legend"] = legend.show
+ if "position" in legend_fields:
+ new_form_data["legendOrientation"] = legend.position
+
+ if config.chart_type == "interactive_pivot":
+ if "temporal_column" in explicit_fields and config.temporal_column is None:
+ new_form_data["granularity_sqla"] = None
+ new_form_data["temporal_columns_lookup"] = None
+ if (
+ "series_limit_metric" in explicit_fields
+ and config.series_limit_metric is None
+ ):
+ new_form_data["series_limit_metric"] = None
+ if "comparison_period" in explicit_fields and config.comparison_period is None:
+ new_form_data["time_compare"] = None
+ if "comparison_type" in explicit_fields and config.comparison_type is None:
+ new_form_data["comparison_type"] = None
+
+ # A Waterfall axis replacement cannot inherit a bucket belonging to the old
+ # temporal subject. Grain omission preserves only while the axis is stable;
+ # explicit null is already handled by the declarative control map above.
+ if (
+ config.chart_type == "waterfall"
+ and existing_form_data.get("x_axis") != new_form_data.get("x_axis")
+ and "time_grain" not in explicit_fields
+ ):
+ new_form_data["time_grain_sqla"] = None
+
+
+def merge_same_viz_form_data(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Preserve saved controls that the typed mapper does not represent.
+
+ The typed MCP surface deliberately models a bounded subset of every Explore
+ control panel. For a replacement within the exact same native ``viz_type``,
+ keys absent from the mapper therefore represent omitted controls and retain
+ their saved values. Mapper output and the chart-specific merge helpers run
+ first and remain authoritative, including explicit empty, false, null, and
+ nested values.
+
+ No generic state crosses a visualization boundary. This prevents query-role
+ keys from the previous plugin (for example ``metric`` or ``groupby``) from
+ leaking into a different plugin whose role contract is unrelated.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if not isinstance(existing_viz_type, str) or existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+
+ _apply_explicit_form_controls(existing_form_data, new_form_data, config)
+
+ for key, value in existing_form_data.items():
+ if key == MCP_DASHBOARD_TIME_FILTER_SUBJECT:
+ # merge_update_form_data owns this provenance marker. Its absence
+ # may be an intentional subject clear and must not be undone by the
+ # generic preservation layer.
+ continue
+ if key not in new_form_data:
+ new_form_data[key] = value
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet state without reclassifying preserved filters.
+
+ The typed Bullet surface intentionally creates only SIMPLE WHERE filters,
+ while saved Explore state may legitimately contain SQL WHERE or SIMPLE
+ HAVING filters. When an update omitted ``filters``, those native objects
+ came from the saved state and are validated by the form-data/query layer;
+ removing them only from this schema-validation copy avoids pretending they
+ were newly supplied typed filters. Explicit filter replacements, including
+ ``[]``, still take the strict native-to-typed path.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ if isinstance(update_config, BulletChartConfig) and (
Review Comment:
Fixed in 703bb6cd. For `update_config is None`, Bullet schema validation now uses a copy without native `adhoc_filters`, while the original byte-faithful SQL/HAVING sequence remains in form data for target-dataset Tier-2 validation and persistence. Direct and public FastMCP dataset-only rebind regressions assert the native sequence and new datasource survive.
##########
tests/unit_tests/mcp_service/chart/test_bullet_chart.py:
##########
@@ -0,0 +1,3380 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Product-path coverage for typed ECharts Bullet MCP support."""
+
+import math
+from datetime import date, datetime, time, timedelta, timezone, tzinfo
+from decimal import Decimal
+from enum import Enum, IntEnum, StrEnum
+from types import SimpleNamespace
+from typing import Any
+from unittest.mock import AsyncMock, MagicMock, patch
+from uuid import UUID
+from zoneinfo import ZoneInfo
+
+import numpy as np
+import pandas as pd
+import pytest
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.zoneinfo import get_zonefile_instance
+from pydantic import TypeAdapter, ValidationError
+
+from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+)
+from superset.mcp_service.chart.chart_utils import (
+ analyze_chart_capabilities,
+ map_bullet_config,
+ map_config_to_form_data,
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ merge_update_form_data,
+ validate_merged_bullet_form_data,
+)
+from superset.mcp_service.chart.compile import _compile_chart
+from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_preview_from_data,
+ _generate_vega_lite_preview_from_data,
+ _javascript_number_string,
+ BulletOutputError,
+ generate_preview_from_form_data,
+ resolve_bullet_render_model,
+)
+from superset.mcp_service.chart.query_result import (
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+)
+from superset.mcp_service.chart.schemas import (
+ ASCIIPreview,
+ BulletChartConfig,
+ ChartConfig,
+ ChartError,
+ DataColumn,
+ GenerateChartRequest,
+ GetChartPreviewRequest,
+ UpdateChartPreviewRequest,
+ UpdateChartRequest,
+ VegaLitePreview,
+ XYChartConfig,
+)
+from superset.mcp_service.chart.tool.generate_chart import generate_chart
+from superset.mcp_service.chart.tool.get_chart_data import (
+ _candidates_single_numeric,
+ _VIZ_CATEGORY,
+)
+from superset.mcp_service.chart.tool.get_chart_preview import (
+ ASCIIPreviewStrategy,
+ TablePreviewStrategy,
+ VegaLitePreviewStrategy,
+)
+from superset.mcp_service.chart.tool.get_chart_type_schema import (
+ _get_chart_type_schema_impl,
+ VALID_CHART_TYPES,
+)
+from superset.mcp_service.chart.tool.update_chart import (
+ _build_preview_form_data,
+ _build_update_payload,
+ update_chart,
+)
+from superset.mcp_service.chart.tool.update_chart_preview import update_chart_preview
+from superset.mcp_service.chart.validation.dataset_validator import DatasetValidator
+from superset.mcp_service.common.error_schemas import DatasetContext
+from superset.utils.json import json_int_dttm_ser
+
+
+def _reject_scalar_conversion(*_args: object, **_kwargs: object) -> Any:
+ raise AssertionError("hostile query scalar method must not run")
+
+
+class _PathHostileStr(str):
+ __getitem__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _PathHostileEnum(str, Enum):
+ FAILED = "warehouse unavailable"
+
+ @property
+ def value(self) -> str:
+ """Reject the public descriptor while preserving Enum's stored value."""
+ return _reject_scalar_conversion()
+
+ __getitem__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileInt(int):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileFloat(float):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileStr(str):
+ __str__ = _reject_scalar_conversion
+ strip = _reject_scalar_conversion
+
+
+class _OutputHostileDecimal(Decimal):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputSafeIntEnum(IntEnum):
+ VALUE = 12
+
+
+class _OutputSafeStrEnum(StrEnum):
+ VALUE = "12.5"
+
+
+_OutputSafeIntEnum.__float__ = _reject_scalar_conversion # type: ignore[method-assign]
+_OutputSafeIntEnum.__str__ = _reject_scalar_conversion # type: ignore[method-assign]
+_OutputSafeStrEnum.__float__ = _reject_scalar_conversion # type: ignore[attr-defined]
+_OutputSafeStrEnum.__str__ = _reject_scalar_conversion # type: ignore[method-assign]
+
+
+def _simple_metric(name: str = "revenue") -> dict[str, str]:
+ return {"name": name, "aggregate": "SUM"}
+
+
+def _tool_user() -> SimpleNamespace:
+ return SimpleNamespace(id=1, username="admin", roles=[], groups=[])
+
+
+def _orm_dataset() -> SimpleNamespace:
+ def column(
+ name: str, type_: str, *, temporal: bool = False, numeric: bool = False
+ ) -> SimpleNamespace:
+ return SimpleNamespace(
+ column_name=name,
+ type=type_,
+ is_temporal=temporal,
+ is_numeric=numeric,
+ is_dttm=temporal,
+ python_date_format=None,
+ )
+
+ return SimpleNamespace(
+ id=7,
+ table_name="sales",
+ schema=None,
+ main_dttm_col="OrderDate",
+ database=SimpleNamespace(database_name="main", db_engine_spec=None),
+ columns=[
+ column("Revenue", "NUMERIC", numeric=True),
+ column("Region", "VARCHAR"),
+ column("Team", "VARCHAR"),
+ column("Status", "VARCHAR"),
+ column("OrderDate", "TIMESTAMP", temporal=True),
+ column("EventDate", "TIMESTAMP", temporal=True),
+ ],
+ metrics=[
+ SimpleNamespace(
+ metric_name="SavedRevenue",
+ expression="SUM(Revenue)",
+ description=None,
+ )
+ ],
+ )
+
+
+def test_bullet_discriminated_union_uses_exact_tag() -> None:
+ config = TypeAdapter(ChartConfig).validate_python(
+ {"chart_type": "bullet", "metric": _simple_metric()}
+ )
+ assert isinstance(config, BulletChartConfig)
+ with pytest.raises(ValidationError):
+ TypeAdapter(ChartConfig).validate_python(
+ {"chart_type": "bullet_chart", "metric": _simple_metric()}
+ )
+
+
+def test_bullet_equal_dimension_aliases_are_order_independent_and_round_trip() -> None:
+ for payload in (
+ {
+ "dimensions": [{"name": "Region", "label": "Market"}, "Team"],
+ "groupby": ["Region", {"column_name": "Team"}],
+ },
+ {
+ "groupby": ["Region", {"column": "Team"}],
+ "dimensions": [{"name": "Region", "label": "Market"}, "Team"],
+ },
+ ):
+ config = BulletChartConfig.model_validate(
+ {"metric": _simple_metric(), **payload}
+ )
+ assert [dimension.name for dimension in config.dimensions or []] == [
+ "Region",
+ "Team",
+ ]
+ mapped = map_bullet_config(config)
+ assert mapped["groupby"] == ["Region", "Team"]
+ round_trip = BulletChartConfig.model_validate(mapped)
+ assert [dimension.name for dimension in round_trip.dimensions or []] == [
+ "Region",
+ "Team",
+ ]
+
+
+@pytest.mark.parametrize(
+ "request_payload",
+ [
+ {"dataset_id": 7},
+ {"identifier": 9},
+ {"dataset_id": 7, "form_data_key": "preview"},
+ ],
+)
+@pytest.mark.parametrize("reverse", [False, True])
+def test_bullet_request_models_reject_conflicting_dimension_aliases(
+ request_payload: dict[str, object], reverse: bool
+) -> None:
+ aliases = [
+ ("dimensions", [{"name": "Region"}, {"name": "Team"}]),
+ ("groupby", ["Team", "Region"]),
+ ]
+ if reverse:
+ aliases.reverse()
+ config = {"chart_type": "bullet", "metric": _simple_metric(), **dict(aliases)}
+ payload = {**request_payload, "config": config}
+ request_type = (
+ UpdateChartRequest
+ if "identifier" in request_payload
+ else (
+ UpdateChartPreviewRequest
+ if "form_data_key" in request_payload
+ else GenerateChartRequest
+ )
+ )
+ with pytest.raises(ValidationError, match="Conflicting Bullet dimension aliases"):
+ request_type.model_validate(payload)
+
+
+@pytest.mark.parametrize(
+ "metric",
+ [
+ {"name": "revenue", "aggregate": "SUM", "label": "Revenue"},
+ {"name": "saved_revenue", "saved_metric": True},
+ {"sql_expression": "SUM(revenue) / COUNT(*)", "label": "Average"},
+ ],
+)
+def test_bullet_accepts_simple_saved_and_sql_metrics(metric: dict[str, object]) -> None:
+ config = BulletChartConfig(metric=metric)
+ form_data = map_bullet_config(config)
+ assert form_data["viz_type"] == "bullet"
+ assert form_data["metric"]
+
+
+def test_bullet_native_form_data_round_trip_is_semantically_stable() -> None:
+ native = {
+ "viz_type": "bullet",
+ "datasource": "7__table",
+ "metric": {
+ "aggregate": "SUM",
+ "column": {"column_name": "Revenue"},
+ "expressionType": "SIMPLE",
+ "label": "Total Revenue",
+ },
+ "groupby": ["Region", "Team"],
+ "ranges": "100,250,500",
+ "range_labels": "Minimum,Target,Stretch",
+ "markers": "300",
+ "marker_labels": "Plan",
+ "marker_lines": "400",
+ "marker_line_labels": "Forecast",
+ "y_axis_format": "$,.0f",
+ "show_labels": False,
+ "show_legend": True,
+ "row_limit": 250,
+ "orderby": [["Region", True], ["Total Revenue", False]],
+ "adhoc_filters": [
+ {
+ "clause": "WHERE",
+ "expressionType": "SIMPLE",
+ "subject": "Status",
+ "operator": "==",
+ "comparator": "Active",
+ }
+ ],
+ }
+ config = BulletChartConfig.model_validate(native)
+ mapped = map_bullet_config(config)
+
+ assert mapped["metric"]["label"] == "Total Revenue"
+ assert mapped["groupby"] == ["Region", "Team"]
+ assert mapped["ranges"] == "100,250,500"
+ assert mapped["range_labels"] == "Minimum,Target,Stretch"
+ assert mapped["markers"] == "300"
+ assert mapped["marker_lines"] == "400"
+ assert mapped["show_labels"] is False
+ assert mapped["show_legend"] is True
+ assert mapped["orderby"][0] == ["Region", True]
+ assert mapped["orderby"][1][0]["label"] == "Total Revenue"
+ assert mapped["adhoc_filters"][0]["subject"] == "Status"
+
+
+def test_bullet_presentation_numbers_use_shortest_round_trip_safe_tokens() -> None:
+ ranges = [1.2345678901234567, 1.7976931348623157e308]
+ markers = [5e-324, -0.0]
+ marker_lines = [9.876543210987654e-200]
+ config = BulletChartConfig(
+ metric=_simple_metric(),
+ ranges=ranges,
+ markers=markers,
+ marker_lines=marker_lines,
+ show_legend=True,
+ )
+ mapped = map_bullet_config(config)
+
+ for key, expected in (
+ ("ranges", ranges),
+ ("markers", markers),
+ ("marker_lines", marker_lines),
+ ):
+ tokens = mapped[key].split(",")
+ assert [float(token) for token in tokens] == expected
+ assert all(
+ float(token).hex() == value.hex()
+ for token, value in zip(tokens, expected, strict=True)
+ )
+
+ round_trip = BulletChartConfig.model_validate(mapped)
+ assert round_trip.ranges == ranges
+ assert round_trip.markers == markers
+ assert round_trip.marker_lines == marker_lines
+
+ model = resolve_bullet_render_model(
+ [{"SUM(revenue)": 1.0}],
+ mapped,
+ )
+ assert model.ranges == ranges
+ assert model.markers == markers
+ assert model.marker_lines == marker_lines
+ assert (
+ "1.7976931348623157e+308"
+ in _generate_ascii_preview_from_data(
+ [{"SUM(revenue)": 1.0}], mapped
+ ).ascii_content
+ )
+ vega = _generate_vega_lite_preview_from_data([{"SUM(revenue)": 1.0}], mapped)
+ assert vega.specification["layer"]
+
+
+def test_bullet_native_saved_metric_and_legacy_metric_aliases() -> None:
+ saved = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": "saved_revenue"}
+ )
+ legacy = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": "sum__revenue"}
+ )
+ assert saved.metric.saved_metric is True
+ assert saved.metric.name == "saved_revenue"
+ assert legacy.metric.name == "sum__revenue"
+ assert legacy.metric.saved_metric is True
+
+
+@pytest.mark.parametrize("metric_name", ["sum__num", "sum__SP_POP_TOTL"])
+@pytest.mark.parametrize(
+ ("request_type", "request_fields"),
+ [
+ (GenerateChartRequest, {"dataset_id": 7}),
+ (UpdateChartRequest, {"identifier": 9}),
+ (UpdateChartPreviewRequest, {"dataset_id": 7}),
+ ],
+)
+def test_bullet_repository_metric_names_round_trip_as_saved_metrics_on_all_requests(
+ metric_name: str,
+ request_type: type[
+ GenerateChartRequest | UpdateChartRequest | UpdateChartPreviewRequest
+ ],
+ request_fields: dict[str, object],
+) -> None:
+ request = request_type.model_validate(
+ {**request_fields, "config": {"chart_type": "bullet", "metric": metric_name}}
+ )
+ config = request.config
+ assert isinstance(config, BulletChartConfig)
+ assert config.metric.saved_metric is True
+ assert map_bullet_config(config)["metric"] == metric_name
+
+
+def test_bullet_legacy_label_only_saved_metric_adapter_is_strict_and_bounded() -> None:
+ config = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": {"label": "sum__num"}}
+ )
+ assert config.metric.saved_metric is True
+ assert map_bullet_config(config)["metric"] == "sum__num"
+
+ with pytest.raises(ValidationError):
+ BulletChartConfig.model_validate(
+ {
+ "viz_type": "bullet",
+ "metric": {"label": "sum__num", "aggregate": "SUM"},
+ }
+ )
+ with pytest.raises(ValidationError, match="at most 255"):
+ BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": {"label": "m" * 256}}
+ )
+
+
+@pytest.mark.parametrize(
+ "metric",
+ [
+ "SavedRevenue",
+ {
+ "aggregate": "SUM",
+ "column": {"column_name": "Revenue"},
+ "expressionType": "SIMPLE",
+ "label": "Simple Revenue",
+ },
+ {
+ "aggregate": None,
+ "column": None,
+ "expressionType": "SQL",
+ "sqlExpression": "SUM(Revenue)",
+ "label": "SQL Revenue",
+ },
+ ],
+)
+def test_bullet_all_metric_shapes_round_trip_full_native_presentation(
+ metric: object,
+) -> None:
+ native = {
+ "viz_type": "bullet",
+ "metric": metric,
+ "groupby": ["Region"],
+ "ranges": "50,100",
+ "range_labels": "Low,High",
+ "markers": "75",
+ "marker_labels": "Plan",
+ "marker_lines": "90",
+ "marker_line_labels": "Forecast",
+ "y_axis_format": "$,.0f",
+ "show_labels": True,
+ "show_legend": True,
+ }
+ mapped = map_bullet_config(BulletChartConfig.model_validate(native))
+ assert validate_merged_bullet_form_data(mapped) is not None
+ assert mapped["groupby"] == ["Region"]
+ assert mapped["ranges"] == "50,100"
+ assert mapped["marker_line_labels"] == "Forecast"
+
+
+def test_bullet_rejects_invalid_roles_and_output_collisions() -> None:
+ with pytest.raises(ValidationError, match="physical dimension"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[{"name": "region", "aggregate": "COUNT"}],
+ )
+ with pytest.raises(ValidationError, match="Duplicate Bullet dimension"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[{"name": "Region"}, {"name": "region"}],
+ )
+ with pytest.raises(ValidationError, match="conflicts with a dimension"):
+ BulletChartConfig(
+ metric={"name": "revenue", "aggregate": "SUM", "label": "Region"},
+ dimensions=[{"name": "region", "label": "Region"}],
+ )
+
+
+def test_bullet_rejects_misaligned_labels_and_bad_order_target() -> None:
+ with pytest.raises(ValidationError, match="one label per ranges"):
+ BulletChartConfig(
+ metric=_simple_metric(), ranges=[1, 2], range_labels=["Only one"]
+ )
+ with pytest.raises(ValidationError, match="unknown: not_a_role"):
+ BulletChartConfig(metric=_simple_metric(), order_by=[{"column": "not_a_role"}])
+
+
+def test_bullet_dimension_labels_are_input_aliases_not_result_aliases() -> None:
+ config = BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Total"},
+ dimensions=[
+ {"name": "Team", "label": "Region"},
+ {"name": "Region", "label": "Market"},
+ ],
+ # The exact physical Region must win over Team's display label.
+ order_by=[
+ {"column": "Region", "ascending": True},
+ {"column": "Revenue", "ascending": False},
+ ],
+ )
+ form_data = map_bullet_config(config)
+ assert form_data["groupby"] == ["Team", "Region"]
+ assert form_data["orderby"] == [
+ ["Region", True],
+ [form_data["metric"], False],
+ ]
+
+ label_order = map_bullet_config(
+ BulletChartConfig(
+ metric=config.metric,
+ dimensions=config.dimensions,
+ order_by=[{"column": "Market"}],
+ )
+ )
+ assert label_order["orderby"] == [["Region", False]]
+
+ model = resolve_bullet_render_model(
+ [{"Team": "Blue", "Region": "North", "Total": 10}], form_data
+ )
+ assert model.dimensions == ["Team", "Region"]
+ assert [model.rows[0][name] for name in model.dimensions] == ["Blue", "North"]
+
+
+def test_bullet_rejects_ambiguous_display_alias_for_ordering() -> None:
+ with pytest.raises(ValidationError, match="ambiguous display alias"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[
+ {"name": "Region", "label": "Area"},
+ {"name": "Team", "label": "area"},
+ ],
+ order_by=[{"column": "AREA"}],
+ )
+
+
+@pytest.mark.parametrize(
+ ("metric", "order_target", "output"),
+ [
+ (
+ {"name": "SavedRevenue", "saved_metric": True, "label": "Friendly"},
+ "Friendly",
+ "SavedRevenue",
+ ),
+ (
+ {"name": "Revenue", "aggregate": "SUM", "label": "Simple Total"},
+ "Revenue",
+ "Simple Total",
+ ),
+ (
+ {"sql_expression": "SUM(Revenue)", "label": "SQL Total"},
+ "SQL Total",
+ "SQL Total",
+ ),
+ ],
+)
+def test_bullet_metric_shapes_share_physical_dimension_output_contract(
+ metric: dict[str, object], order_target: str, output: str
+) -> None:
+ config = BulletChartConfig(
+ metric=metric,
+ dimensions=[{"name": "Region", "label": "Market"}],
+ order_by=[{"column": order_target}],
+ )
+ form_data = map_bullet_config(config)
+ assert form_data["groupby"] == ["Region"]
+ assert form_data["orderby"] == [[form_data["metric"], False]]
+ model = resolve_bullet_render_model([{"Region": "North", output: 12}], form_data)
+ assert model.metric_field == output
+ assert model.dimensions == ["Region"]
+
+
+def test_bullet_mapper_preserves_omission_and_honors_explicit_values() -> None:
+ omitted = map_bullet_config(BulletChartConfig(metric=_simple_metric()))
+ explicit = map_bullet_config(
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[],
+ filters=[],
+ ranges=[],
+ show_labels=False,
+ show_legend=False,
+ row_limit=42,
+ time_range=None,
+ )
+ )
+ for key in (
+ "groupby",
+ "adhoc_filters",
+ "ranges",
+ "show_labels",
+ "show_legend",
+ "row_limit",
+ "time_range",
+ ):
+ assert key not in omitted
+ assert explicit["groupby"] == []
+ assert explicit["adhoc_filters"] == []
+ assert explicit["ranges"] == ""
+ assert explicit["show_labels"] is False
+ assert explicit["show_legend"] is False
+ assert explicit["row_limit"] == 42
+ assert explicit["time_range"] is None
+
+
+def test_bullet_registry_schema_and_recommendation_metadata() -> None:
+ from superset.mcp_service.app import get_default_instructions
+ from superset.mcp_service.chart.registry import display_name_for_viz_type, get
+
+ plugin = get("bullet")
+ assert plugin is not None
+ assert plugin.resolve_viz_type(None) == "bullet"
+ assert display_name_for_viz_type("bullet") == "Bullet Chart"
+ assert "bullet" in VALID_CHART_TYPES
+ discovered = _get_chart_type_schema_impl("bullet")
+ assert discovered["chart_type"] == "bullet"
+ assert discovered["examples"][0]["ranges"] == [100000, 250000, 500000]
+ assert _VIZ_CATEGORY["bullet"] == "bullet"
+ candidates = _candidates_single_numeric(
+ DataColumn(
+ name="Revenue",
+ display_name="Revenue",
+ data_type="numeric",
+ sample_values=[1],
+ null_count=0,
+ unique_count=1,
+ ),
+ row_count=1,
+ )
+ assert "bullet chart" in candidates
+ guidance = get_default_instructions()
+ assert 'chart_type="bullet": Bullet Chart' in guidance
+ assert "waterfall, bullet, and interactive_pivot" in guidance
+
+
+def test_bullet_dataset_normalization_canonicalizes_every_reference() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "Region", "type": "VARCHAR"},
+ {"name": "OrderDate", "type": "TIMESTAMP", "is_temporal": True},
+ {"name": "Status", "type": "VARCHAR"},
+ ],
+ available_metrics=[],
+ )
+ config = BulletChartConfig(
+ metric={"name": "revenue", "aggregate": "SUM"},
+ dimensions=[{"name": "region"}],
+ temporal_column="orderdate",
+ filters=[{"column": "status", "op": "=", "value": "active"}],
+ order_by=[{"column": "region", "ascending": True}],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ normalized = plugin.normalize_column_refs(config, context)
+ assert normalized.metric.name == "Revenue"
+ assert normalized.dimensions[0].name == "Region"
+ assert normalized.temporal_column == "OrderDate"
+ assert normalized.filters[0].column == "Status"
+ assert normalized.order_by[0].column == "Region"
+ assert normalized.model_fields_set == config.model_fields_set
+
+
+def test_bullet_dataset_normalization_rejects_ambiguous_casefold_candidates() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "revenue", "type": "NUMERIC", "is_numeric": True},
+ ],
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ config = BulletChartConfig(metric={"name": "REVENUE", "aggregate": "SUM"})
+ with pytest.raises(ValueError, match="Revenue, revenue"):
+ plugin.normalize_column_refs(config, context)
+
+
+def test_bullet_numeric_output_constraint_rejects_text_min() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[{"name": "status", "type": "VARCHAR"}],
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ config = BulletChartConfig(metric={"name": "status", "aggregate": "MIN"})
+ with patch.object(DatasetValidator, "_get_dataset_context", return_value=context):
+ error = plugin.post_map_validate(config, {}, dataset_id=7)
+ assert error is not None
+ assert error.error_type == "non_numeric_bullet_metric"
+
+
+@pytest.mark.parametrize("reverse_metadata", [False, True])
+def test_bullet_exact_case_type_and_role_resolution_is_order_independent(
+ reverse_metadata: bool,
+) -> None:
+ from superset.mcp_service.chart.registry import get
+
+ columns = [
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "revenue", "type": "VARCHAR", "is_numeric": False},
+ {"name": "Region", "type": "VARCHAR"},
+ ]
+ if reverse_metadata:
+ columns.reverse()
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=columns,
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+
+ numeric = BulletChartConfig(metric={"name": "Revenue", "aggregate": "SUM"})
+ text = BulletChartConfig(metric={"name": "revenue", "aggregate": "MIN"})
+ with patch.object(DatasetValidator, "_get_dataset_context", return_value=context):
+ assert plugin.post_map_validate(numeric, {}, dataset_id=7) is None
+ error = plugin.post_map_validate(text, {}, dataset_id=7)
+ assert error is not None
+ assert error.error_type == "non_numeric_bullet_metric"
+
+ roles = BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM"},
+ dimensions=[{"name": "revenue"}, {"name": "Region"}],
+ filters=[{"column": "revenue", "op": "=", "value": "retail"}],
+ order_by=[{"column": "revenue", "ascending": True}],
+ )
+ normalized = plugin.normalize_column_refs(roles, context)
+ assert normalized.metric.name == "Revenue"
+ assert [dimension.name for dimension in normalized.dimensions or []] == [
+ "revenue",
+ "Region",
+ ]
+ assert normalized.filters
+ assert normalized.filters[0].column == "revenue"
+ assert normalized.order_by[0].column == "revenue"
+
+ ambiguous = BulletChartConfig(metric={"name": "REVENUE", "aggregate": "SUM"})
+ with pytest.raises(ValueError, match="Ambiguous"):
+ plugin.normalize_column_refs(ambiguous, context)
+
+
+@pytest.mark.parametrize("reverse_metadata", [False, True])
+def test_generic_aggregation_validation_uses_exact_case_before_type(
+ reverse_metadata: bool,
+) -> None:
+ from superset.mcp_service.chart.schemas import PieChartConfig
+
+ columns = [
+ {"name": "Amount", "type": "BIGINT", "is_numeric": True},
+ {"name": "amount", "type": "VARCHAR", "is_numeric": False},
+ ]
+ if reverse_metadata:
+ columns.reverse()
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=columns,
+ available_metrics=[],
+ )
+
+ assert (
+ DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "Amount", "aggregate": "SUM"}).metric],
+ context,
+ )
+ == []
+ )
+ errors = DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "amount", "aggregate": "SUM"}).metric],
+ context,
+ )
+ assert errors
+ assert errors[0].error_type == "invalid_aggregation"
+
+ ambiguous = DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "AMOUNT", "aggregate": "SUM"}).metric],
+ context,
+ )
+ assert ambiguous
+ assert ambiguous[0].error_type == "ambiguous_column_reference"
+
+ valid, error = DatasetValidator.validate_against_dataset(
+ PieChartConfig(
+ dimension={"name": "amount"},
+ metric={"name": "Amount", "aggregate": "SUM"},
+ ),
+ 7,
+ dataset_context=context,
+ )
+ assert valid is True
+ assert error is None
+
+
+@pytest.mark.parametrize(
+ ("metric", "field"),
+ [
+ ({"name": "Revenue", "aggregate": "SUM", "label": "Simple"}, "Simple"),
+ ({"name": "SavedRevenue", "saved_metric": True}, "SavedRevenue"),
+ ({"sql_expression": "SUM(Revenue)", "label": "SQL Total"}, "SQL Total"),
+ ],
+)
+def test_bullet_result_roles_are_exact_for_every_metric_shape(
+ metric: dict[str, object], field: str
+) -> None:
+ form_data = map_bullet_config(BulletChartConfig(metric=metric))
+ model = resolve_bullet_render_model([{field.swapcase(): "12.5"}], form_data)
+ assert model.metric_field == field.swapcase()
+ assert model.measures == [12.5]
+
+
+@pytest.mark.parametrize(
+ "rows, message",
+ [
+ ([{"other": 123}], "missing"),
+ ([{"Revenue": "not a number"}], "non-numeric text"),
+ ([{"Revenue": math.nan}], "NaN or infinite"),
+ ([{"Revenue": math.inf}], "NaN or infinite"),
+ ([{"Revenue": 1}, {}], "row 1.*missing"),
+ ([{"REVENUE": 1, "revenue": 2}], "ambiguous"),
+ ],
+)
+def test_bullet_result_validation_rejects_malformed_rows(
+ rows: list[dict[str, object]], message: str
+) -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "amount", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ with pytest.raises(BulletOutputError, match=message):
+ resolve_bullet_render_model(rows, form_data)
+
+
+def test_bullet_result_validation_accepts_null_and_numeric_strings() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "amount", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Region"}],
+ )
+ )
+ model = resolve_bullet_render_model(
+ [
+ {"Region": "North", "Revenue": None},
+ {"Region": "South", "Revenue": " 4.25 "},
+ ],
+ form_data,
+ )
+ assert model.measures == [0.0, 4.25]
+
+
+@pytest.mark.parametrize(
+ ("presentation", "message"),
+ [
+ ({"ranges": "10,nope"}, r"ranges\[1\].*not numeric"),
+ ({"markers": "NaN"}, r"markers\[0\].*NaN or infinite"),
+ (
+ {"ranges": "10,20", "range_labels": "Only one"},
+ "one label per value",
+ ),
+ ],
+)
+def test_bullet_result_validation_rejects_malformed_presentation(
+ presentation: dict[str, object], message: str
+) -> None:
+ form_data = {
+ **map_bullet_config(
+ BulletChartConfig(metric={"name": "amount", "aggregate": "SUM"})
+ ),
+ **presentation,
+ }
+ with pytest.raises(BulletOutputError, match=message):
+ resolve_bullet_render_model([{"SUM(amount)": 1}], form_data)
+
+
+def test_bullet_compile_accepts_empty_ungrouped_result() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {"queries": [{"data": []}]}
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ assert result.success is True
+ assert result.row_count == 0
+
+
+def test_bullet_compile_inspects_top_level_and_query_error_envelopes() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(metric={"name": "Revenue", "aggregate": "SUM"})
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {
+ "status": "success",
+ "queries": [{"status": "failed", "message": "warehouse timeout"}],
+ }
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ assert result.success is False
+ assert "warehouse timeout" in (result.error or "")
+
+
+_MALFORMED_QUERY_ENVELOPES: list[object] = [
+ None,
+ [],
+ {},
+ {"queries": None},
+ {"queries": []},
+ {"queries": [None]},
+ {"queries": [{}]},
+ {"queries": [{"data": None}]},
+ {"queries": [{"data": []}, {"data": "not-an-array"}]},
+ {
+ "queries": [
+ {
+ "data": [{"Revenue": 12}],
+ "colnames": ["Revenue"],
+ "coltypes": [],
+ }
+ ]
+ },
+]
+
+
+def _compile_bullet_with_result(result: object) -> Any:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = result
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ return _compile_chart(form_data, 7)
+
+
+@pytest.mark.parametrize("envelope", _MALFORMED_QUERY_ENVELOPES)
+def test_bullet_compile_returns_stable_error_for_malformed_envelopes(
+ envelope: object,
+) -> None:
+ result = _compile_bullet_with_result(envelope)
+ assert result.success is False
+ assert result.error_code == "CHART_COMPILE_FAILED"
+ assert result.error_obj is not None
+ assert result.error_obj.error_type == "compile_error"
+
+
+@pytest.mark.parametrize(
+ ("data", "expected_code", "expected_type"),
+ [
+ ([1], "CHART_COMPILE_FAILED", "compile_error"),
+ (
+ [{"Revenue": 10**10000}],
+ "CHART_COMPILE_FAILED",
+ "compile_error",
+ ),
+ ],
+)
+def test_bullet_compile_returns_malformed_output_for_bad_rows(
+ data: list[object],
+ expected_code: str,
+ expected_type: str,
+) -> None:
+ result = _compile_bullet_with_result({"queries": [{"data": data}]})
+ assert result.success is False
+ assert result.error_code == expected_code
+ assert result.error_obj is not None
+ assert result.error_obj.error_type == expected_type
+
+
+def test_bullet_shared_query_builder_matches_frontend_build_query() -> None:
+ metric = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Region"}],
+ row_limit=25,
+ order_by=[{"column": "Revenue", "ascending": False}],
+ )
+ )
+ with patch(
+ "superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
+ return_value="base",
+ ):
+ queries = build_query_dicts_from_form_data(metric, 7, "table")
+ assert len(queries) == 1
+ assert queries[0]["columns"] == ["Region"]
+ assert queries[0]["metrics"] == [metric["metric"]]
+ assert queries[0]["orderby"] == metric["orderby"]
+ assert queries[0]["row_limit"] == 25
+
+
+def test_bullet_compile_path_uses_groupby_metric_orderby_and_usable_result() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Region"}],
+ order_by=[{"column": "Revenue", "ascending": False}],
+ )
+ )
+ context = MagicMock()
+ factory = MagicMock()
+ factory.create.return_value = context
+ command = MagicMock()
+ command.run.return_value = {
+ "queries": [{"data": [{"Region": "North", "Revenue": "12.5"}]}]
+ }
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ query = factory.create.call_args.kwargs["queries"][0]
+ assert query["columns"] == ["Region"]
+ assert query["metrics"] == [form_data["metric"]]
+ assert query["orderby"] == form_data["orderby"]
+ assert result.success is True
+ assert result.row_count == 1
+
+
+def test_bullet_compile_projects_dataframe_timestamp_before_validation() -> None:
+ from superset.dataframe import df_to_records
+
+ dublin = dateutil_tz.gettz("Europe/Dublin")
+ new_york = dateutil_tz.gettz("America/New_York")
+ assert dublin is not None
+ assert new_york is not None
+ source_values = [
+ pd.Timestamp("2024-01-02 03:04:05.123456789"),
+ datetime(2024, 10, 27, 1, 30, tzinfo=dublin, fold=1),
+ datetime(2024, 3, 10, 2, 30, tzinfo=new_york),
+ datetime(2040, 7, 1, 12, tzinfo=new_york),
+ ]
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Category"}],
+ )
+ )
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {"queries": [{"data": rows}]}
+ captured: list[list[dict[str, Any]]] = []
+
+ def validate(data: list[dict[str, Any]], config: dict[str, Any]) -> Any:
+ captured.append(data)
+ return resolve_bullet_render_model(data, config)
+
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ patch(
+ "superset.mcp_service.chart.preview_utils.resolve_bullet_render_model",
+ side_effect=validate,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+
+ assert result.success is True
+ assert [row["Category"] for row in captured[0]] == [
+ 1704164645123.456,
+ 1729989000000.0,
+ 1710052200000.0,
+ 2224774800000.0,
+ ]
+
+
+def test_bullet_compile_projects_real_dataframe_durations_to_chart_data_wire() -> None:
+ from superset.commands.chart.data.get_data_command import ChartDataCommand
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+ from superset.utils import json
+
+ source_values = [
+ timedelta(0),
+ timedelta(days=1, seconds=2, microseconds=3),
+ pd.Timedelta(-1, unit="ns"),
+ pd.Timedelta("1 days 00:00:02.000003004"),
+ np.timedelta64(123456789, "ns"),
+ np.timedelta64("NaT"),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Duration": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+ expected = [
+ None
+ if row["Duration"] is None
+ else json.loads(json.dumps(row["Duration"], default=json.json_int_dttm_ser))
+ for row in rows
+ ]
Review Comment:
Reproduced; no product/test crash occurs. `df_to_records(pd.DataFrame(...))` promotes exact `np.timedelta64(123456789, "ns")` to exact `pd.Timedelta` and converts `np.timedelta64("NaT")` to `None` before this expected-value calculation. 703bb6cd adds those producer-boundary assertions immediately before the existing `json_int_dttm_ser` parity check. The test passes in the 1,451-test focused run and 4,386-test MCP run.
##########
tests/unit_tests/mcp_service/chart/tool/test_get_chart_preview.py:
##########
@@ -45,12 +53,1126 @@
_first_query_has_fields,
_no_query_fields_error,
ASCIIPreviewStrategy,
+ get_chart_preview,
PreviewFormatStrategy,
TablePreviewStrategy,
)
from superset.utils import json as utils_json
+def _query_context_stub(form_data: dict[str, Any] | None = None) -> Any:
+ """Return the minimal real-shaped context needed by Jinja form-data seeding."""
+ return SimpleNamespace(form_data=form_data or {}, queries=[])
+
+
+def _entrypoint_preview(content: str) -> ChartPreview:
+ return ChartPreview(
+ chart_id=1,
+ chart_name="",
+ chart_type="bullet",
+ explore_url="",
+ content=ASCIIPreview(ascii_content=content, width=80, height=20),
+ chart_description="",
+ accessibility=AccessibilityMetadata(
+ color_blind_safe=True,
+ alt_text="",
+ high_contrast_available=False,
+ ),
+ performance=PerformanceMetadata(
+ query_duration_ms=0,
+ cache_status="miss",
+ optimization_suggestions=[],
+ ),
+ )
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ ("request_payload", "extra", "expected_type"),
+ [
+ ({"id": 1, "format": "ascii"}, 0, ChartPreview),
+ ({"form_data_key": "cached-preview", "format": "ascii"}, 1, ChartError),
+ ],
+ ids=["identifier-alias-exact-limit", "cached-preview-limit-plus-one"],
+)
+async def test_get_chart_preview_entrypoint_preflights_complete_exact_wire_response(
+ request_payload: dict[str, object], extra: int, expected_type: type[object]
+) -> None:
+ empty = _entrypoint_preview("")
+ filler = "x" * (
+ MAX_QUERY_RESULT_VALUE_BYTES - len(empty.model_dump_json().encode()) + extra
+ )
+ candidate = _entrypoint_preview(filler)
+ request = GetChartPreviewRequest.model_validate(request_payload)
+ ctx = MagicMock()
+ ctx.info = AsyncMock()
+ ctx.debug = AsyncMock()
+ ctx.warning = AsyncMock()
+
+ user = MagicMock(id=1, username="admin", roles=[], groups=[])
+ with (
+ patch("superset.mcp_service.auth.get_user_from_request", return_value=user),
+ patch(
+ "superset.mcp_service.chart.tool.get_chart_preview."
+ "_get_chart_preview_internal",
+ new=AsyncMock(return_value=candidate),
+ ),
+ ):
+ result = await get_chart_preview(request, ctx=ctx)
+
+ assert isinstance(result, expected_type)
+ if extra == 0:
+ assert len(candidate.model_dump_json().encode()) == (
+ MAX_QUERY_RESULT_VALUE_BYTES
+ )
+ assert result is candidate
+ else:
+ assert isinstance(result, ChartError)
+ assert result.error_type == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("format_", ["ascii", "vega_lite"])
+async def test_bullet_numeric_and_temporal_categories_reach_real_mcp_entrypoint(
+ format_: str,
+) -> None:
+ from contextlib import nullcontext
+
+ from fastmcp import Client
+
+ from superset.mcp_service.app import mcp
+
+ preview_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_preview"
+ )
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=121,
+ slice_name="Number boundaries",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ params=utils_json.dumps(form_data),
+ )
+ rows = [
+ {"Category": 9007199254740993, "Revenue": 1},
+ {"Category": Decimal("1.0000000000000001"), "Revenue": 2},
+ {"Category": Decimal("1.7976931348623159e308"), "Revenue": 3},
+ {"Category": date(2026, 9, 2), "Revenue": 4},
+ {
+ "Category": datetime(2026, 9, 2, 3, 4, 5, tzinfo=timezone.utc),
+ "Revenue": 5,
+ },
+ {
+ "Category": datetime(
+ 2023,
+ 11,
+ 5,
+ 1,
+ 30,
+ tzinfo=ZoneInfo("America/New_York"),
+ fold=0,
+ ),
+ "Revenue": 6,
+ },
+ {
+ "Category": datetime(
+ 2023,
+ 11,
+ 5,
+ 1,
+ 30,
+ tzinfo=ZoneInfo("America/New_York"),
+ fold=1,
+ ),
+ "Revenue": 7,
+ },
+ ]
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ }
+ ]
+ }
+
+ query_context = SimpleNamespace(
+ form_data={},
+ queries=[SimpleNamespace(metrics=["Revenue"], columns=["Category"])],
+ )
+ user = MagicMock(id=1, username="admin", roles=[], groups=[])
+ with (
+ patch("superset.mcp_service.auth.get_user_from_request", return_value=user),
+ patch("superset.mcp_service.auth.check_tool_permission", return_value=True),
+ patch.object(preview_module, "find_chart_by_identifier", return_value=chart),
+ patch.object(preview_module.db.session, "refresh", return_value=None),
+ patch.object(
+ preview_module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch.object(
+ preview_module.event_logger,
+ "log_context",
+ side_effect=lambda **_kwargs: nullcontext(),
+ ),
+ patch.object(
+ preview_module,
+ "build_query_context_from_form_data",
+ return_value=query_context,
+ ),
+ patch.object(preview_module, "set_query_context_form_data", return_value=None),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ patch.object(
+ preview_module, "get_superset_base_url", return_value="http://localhost"
+ ),
+ ):
+ async with Client(mcp) as client:
+ result = await client.call_tool(
+ "get_chart_preview",
+ {"request": {"id": 121, "format": format_}},
+ )
+
+ payload = utils_json.loads(result.content[0].text)
+ if format_ == "ascii":
+ content = payload["content"]["ascii_content"]
+ assert "9007199254740992" in content
+ assert "Infinity" in content
+ assert "1788307200000" in content
+ assert "1699162200000" in content
+ assert "1699165800000" in content
+ else:
+ specification = payload["content"]["specification"]
+ bar = next(
+ layer for layer in specification["layer"] if layer["mark"]["type"] == "bar"
+ )
+ category_field = bar["encoding"]["y"]["field"]
+ assert [row[category_field] for row in specification["data"]["values"]] == [
+ "9007199254740992",
+ "1",
+ "Infinity",
+ "1788307200000",
+ "1788318245000",
+ "1699162200000",
+ "1699165800000",
+ ]
+ assert [row["Category"] for row in specification["data"]["values"][3:]] == [
+ 1788307200000.0,
+ 1788318245000.0,
+ 1699162200000.0,
+ 1699165800000.0,
+ ]
+ assert bar["encoding"]["tooltip"][0]["field"] == category_field
+ assert "transform" not in specification
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("format_", ["ascii", "vega_lite"])
+async def test_bullet_timestamp_categories_from_dataframe_reach_fastmcp(
+ format_: str,
+) -> None:
+ from contextlib import nullcontext
+
+ from fastmcp import Client
+
+ from superset.dataframe import df_to_records
+ from superset.mcp_service.app import mcp
+
+ preview_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_preview"
+ )
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ zoneinfo_tz = ZoneInfo("America/New_York")
+ pytz_tz = pytz.timezone("America/New_York")
+ dateutil_dublin = dateutil_tz.gettz("Europe/Dublin")
+ dateutil_new_york = dateutil_tz.gettz("America/New_York")
+ assert dateutil_dublin is not None
+ assert dateutil_new_york is not None
+ dublin_fold = datetime(
+ 2024,
+ 10,
+ 27,
+ 1,
+ 30,
+ 0,
+ 123456,
+ tzinfo=dateutil_dublin,
+ fold=1,
+ )
+ new_york_gap = datetime(2024, 3, 10, 2, 30, 0, 123456, tzinfo=dateutil_new_york)
+ source_values = [
+ pd.Timestamp("2024-01-02 03:04:05.123456789"),
+ pd.Timestamp("2024-01-02 08:34:05.123456789+05:30"),
+ pd.Timestamp(datetime(2024, 11, 3, 1, 30, tzinfo=zoneinfo_tz, fold=0)),
+ pd.Timestamp(datetime(2024, 11, 3, 1, 30, tzinfo=zoneinfo_tz, fold=1)),
+ pd.Timestamp(pytz_tz.localize(datetime(2024, 11, 3, 1, 30), is_dst=True)),
+ pd.Timestamp(pytz_tz.localize(datetime(2024, 11, 3, 1, 30), is_dst=False)),
+ pd.Timestamp("1969-12-31 23:59:59.999999999"),
+ date(2024, 1, 2),
+ pd.NaT,
+ dublin_fold,
+ new_york_gap,
+ datetime(2040, 7, 1, 12, 0, 0, 123456, tzinfo=dateutil_new_york),
+ datetime(
+ 2024,
+ 3,
+ 10,
+ 2,
+ 30,
+ 0,
+ 123456,
+ tzinfo=dateutil_tz.tzoffset("EDT", -4 * 3600),
+ ),
+ datetime(2024, 3, 10, 6, 30, 0, 123456, tzinfo=timezone.utc),
+ pd.Timestamp(dublin_fold),
+ pd.Timestamp(new_york_gap),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+ if format_ == "ascii":
+ # ASCII intentionally displays at most ten categories. Keep every
+ # dateutil named-zone edge in this public-format invocation.
+ rows = rows[9:12]
+ expected = [
+ "1704164645123.456",
+ "1704164645123.456",
+ "1730611800000",
+ "1730615400000",
+ "1730611800000",
+ "1730615400000",
+ "-0.0010000000000287557",
+ "1704153600000",
+ "null",
+ "1729989000123.456",
+ "1710052200123.456",
+ "2224774800123.456",
+ "1710052200123.456",
+ "1710052200123.456",
+ "1729989000123.456",
+ "1710052200123.456",
+ ]
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=122,
+ slice_name="Timestamp categories",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ params=utils_json.dumps(form_data),
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ }
+ ]
+ }
+
+ query_context = SimpleNamespace(
+ form_data={},
+ queries=[SimpleNamespace(metrics=["Revenue"], columns=["Category"])],
+ )
+ user = MagicMock(id=1, username="admin", roles=[], groups=[])
+ with (
+ patch("superset.mcp_service.auth.get_user_from_request", return_value=user),
+ patch("superset.mcp_service.auth.check_tool_permission", return_value=True),
+ patch.object(preview_module, "find_chart_by_identifier", return_value=chart),
+ patch.object(preview_module.db.session, "refresh", return_value=None),
+ patch.object(
+ preview_module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch.object(
+ preview_module.event_logger,
+ "log_context",
+ side_effect=lambda **_kwargs: nullcontext(),
+ ),
+ patch.object(
+ preview_module,
+ "build_query_context_from_form_data",
+ return_value=query_context,
+ ),
+ patch.object(preview_module, "set_query_context_form_data", return_value=None),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ patch.object(
+ preview_module, "get_superset_base_url", return_value="http://localhost"
+ ),
+ ):
+ async with Client(mcp) as client:
+ result = await client.call_tool(
+ "get_chart_preview",
+ {"request": {"id": 122, "format": format_}},
+ )
+
+ payload = utils_json.loads(result.content[0].text)
+ if format_ == "ascii":
+ content = payload["content"]["ascii_content"]
+ for category in set(expected[9:12]):
+ assert category[:20] in content
+ else:
+ specification = payload["content"]["specification"]
+ bar = next(
+ layer for layer in specification["layer"] if layer["mark"]["type"] == "bar"
+ )
+ category_field = bar["encoding"]["y"]["field"]
+ assert [row[category_field] for row in specification["data"]["values"]] == (
+ expected
+ )
+ assert bar["encoding"]["tooltip"][0]["field"] == category_field
+ assert [row["Category"] for row in specification["data"]["values"]] == [
+ 1704164645123.456,
+ 1704164645123.456,
+ 1730611800000.0,
+ 1730615400000.0,
+ 1730611800000.0,
+ 1730615400000.0,
+ -0.0010000000000287557,
+ 1704153600000.0,
+ None,
+ 1729989000123.456,
+ 1710052200123.456,
+ 2224774800123.456,
+ 1710052200123.456,
+ 1710052200123.456,
+ 1729989000123.456,
+ 1710052200123.456,
+ ]
+
+
+@pytest.mark.asyncio
+async def test_transitionless_dateutil_dataframe_reaches_fastmcp_preview() -> None:
+ from contextlib import nullcontext
+
+ from fastmcp import Client
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+ from superset.mcp_service.app import mcp
+ from superset.mcp_service.chart.preview_utils import _javascript_number_string
+ from superset.utils.json import json_int_dttm_ser
+
+ preview_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_preview"
+ )
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ names = [
+ "UTC",
+ "GMT",
+ "Universal",
+ "Zulu",
+ "EST",
+ "HST",
+ "MST",
+ "Etc/GMT+1",
+ "Etc/GMT-2",
+ ]
+ values = []
+ for getter in (dateutil_tz.gettz, get_zonefile_instance().get):
+ for name in names:
+ timezone_value = getter(name)
+ assert timezone_value is not None
+ values.append(
+ datetime(2040, 7, 1, 12, 34, 56, 123456, tzinfo=timezone_value)
+ )
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(values, dtype=object),
+ "Revenue": range(1, len(values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+
+ class _ProducerContext:
+ result_type = ChartDataResultType.FULL
+
+ def get_payload(self, **_kwargs: Any) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ producer_result = ProducerChartDataCommand(
+ _ProducerContext() # type: ignore[arg-type]
+ ).run()
+ expected_numbers = [json_int_dttm_ser(value) for value in values]
+ expected_categories = [
+ _javascript_number_string(float(value)) for value in expected_numbers
+ ]
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=123,
+ slice_name="Transitionless timestamps",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ params=utils_json.dumps(form_data),
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return producer_result
+
+ query_context = SimpleNamespace(
+ form_data={},
+ queries=[SimpleNamespace(metrics=["Revenue"], columns=["Category"])],
+ )
+ user = MagicMock(id=1, username="admin", roles=[], groups=[])
+ with (
+ patch("superset.mcp_service.auth.get_user_from_request", return_value=user),
+ patch("superset.mcp_service.auth.check_tool_permission", return_value=True),
+ patch.object(preview_module, "find_chart_by_identifier", return_value=chart),
+ patch.object(preview_module.db.session, "refresh", return_value=None),
+ patch.object(
+ preview_module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch.object(
+ preview_module.event_logger,
+ "log_context",
+ side_effect=lambda **_kwargs: nullcontext(),
+ ),
+ patch.object(
+ preview_module,
+ "build_query_context_from_form_data",
+ return_value=query_context,
+ ),
+ patch.object(preview_module, "set_query_context_form_data", return_value=None),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ patch.object(
+ preview_module, "get_superset_base_url", return_value="http://localhost"
+ ),
+ ):
+ async with Client(mcp) as client:
+ result = await client.call_tool(
+ "get_chart_preview",
+ {"request": {"id": 123, "format": "vega_lite"}},
+ )
+
+ payload = utils_json.loads(result.content[0].text)
+ specification = payload["content"]["specification"]
+ bar = next(
+ layer for layer in specification["layer"] if layer["mark"]["type"] == "bar"
+ )
+ category_field = bar["encoding"]["y"]["field"]
+ assert [row["Category"] for row in specification["data"]["values"]] == (
+ expected_numbers
+ )
+ assert [row[category_field] for row in specification["data"]["values"]] == (
+ expected_categories
+ )
+ assert bar["encoding"]["tooltip"][0]["field"] == category_field
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("format_", ["ascii", "vega_lite"])
+async def test_duration_dataframe_reaches_fastmcp_bullet_preview(
+ format_: str,
+) -> None:
+ from contextlib import nullcontext
+
+ import numpy as np
+ from fastmcp import Client
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+ from superset.mcp_service.app import mcp
+
+ preview_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_preview"
+ )
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ source_values = [
+ timedelta(0),
+ timedelta(days=1, seconds=2, microseconds=3),
+ pd.Timedelta(-1, unit="ns"),
+ np.timedelta64(123456789, "ns"),
+ np.timedelta64("NaT"),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Duration": pd.Series(source_values, dtype=object),
+ "Revenue": [50, 120, 350, 0, 200],
+ }
+ ),
+ convert_big_integers=False,
+ )
+ expected = [
+ "null"
+ if row["Duration"] is None
+ else utils_json.loads(
+ utils_json.dumps(row["Duration"], default=utils_json.json_int_dttm_ser)
+ )
+ for row in rows
+ ]
Review Comment:
Reproduced; the premise is stale for rows produced by `df_to_records`: exact NumPy duration becomes exact `pd.Timedelta`, and NumPy NaT becomes `None`. 703bb6cd adds explicit assertions at that boundary in the public FastMCP preview regression before the serializer-derived expected values. The test passes locally and in the full MCP suite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5511638243
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4224297719
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2412,101 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ escape_separator(label)
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(label + FLAT_COLUMN_SEPARATOR)
Review Comment:
Confirmed: with `truncate_metric: true` and one metric the rename step leaves `{event_date, East, West}` and field discovery found nothing. Fixed in f19daaf51a99f657d599f3f02dd0944d9546624a: when no metric-prefixed field matches, there is a single metric, and `truncate_metric` is set, every non-x-axis column is folded as a category series. New `test_xy_preview_renders_truncated_metric_series` runs the real post-processing chain and asserts the fold covers `East` and `West`; it fails without the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5785731216
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3974850153
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1007,3 +1007,19 @@ The callable must be cheap and in-process (consult already-loaded feature flags
- **[MCP Integration](/developer-docs/extensions/mcp)** -- Build custom MCP tools and prompts via Superset extensions
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
Review Comment:
Adding Bullet compatibility here leaves the supported-plugin list above omitting `bullet`, even though this PR registers it. Operators can read the configuration guide as saying the server does not support the chart type. Could the list include `bullet`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5736821658
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4245121437
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -3538,7 +3570,660 @@ def _as_column_list(value: Any) -> list[Any]:
return [value]
-def analyze_chart_capabilities(viz_type: str | None, config: Any) -> ChartCapabilities:
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ # A missing saved viz_type is a boundary too, matching merge_chart_form_data.
+ if existing_form_data.get("viz_type") != new_form_data.get("viz_type"):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+ if (
+ existing_binding is None
+ and existing_subject is None
+ and range_explicit
+ and not subject_authoritative
+ and "filters" not in explicit_fields
+ ):
+ # A range-only update on an unmarked chart adopts its saved temporal
+ # filter even when Explore bound it to a column other than the mapper's
+ # proposed subject. Zero or several native ranges stay ambiguous, so the
+ # proposed binding is added instead of guessing which one to rewrite.
+ unmarked_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("expressionType") == "SIMPLE"
+ and filter_.get("clause") == "WHERE"
+ and isinstance(filter_.get("subject"), str)
+ ]
+ if len(unmarked_matches) == 1:
+ existing_binding = unmarked_matches[0]
+ existing_subject = existing_binding["subject"]
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ elif range_explicit:
+ # With no saved binding to rewrite, an explicit range uses the
+ # mapper's generated binding rather than being dropped.
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+# The shared preservation registry may carry envelope, presentation, and time
+# keys over from another visualization. Bullet does not model them, so they are
+# opaque persisted state rather than misspelled typed fields.
+_BULLET_UNMODELED_PRESERVED_KEYS = (
+ FORM_DATA_UPDATE_PRESERVE_KEYS["envelope"]
+ | FORM_DATA_UPDATE_PRESERVE_KEYS["presentation"]
+ | FORM_DATA_UPDATE_PRESERVE_KEYS["time"]
+) - _get_known_fields(BulletChartConfig)
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet controls without reinterpreting inherited query roles.
+
+ Saved Explore state may contain SQL dimensions and SQL WHERE/SIMPLE HAVING
+ filters beyond the typed authoring surface. Omitted roles are validated by
+ the native query contract and compilation, not as newly authored physical
+ columns or SIMPLE WHERE filters. Only this validation copy excludes them
+ and native query metadata; compiled and persisted form data stays intact.
+ Explicit replacements, including ``[]``, retain strict typed validation.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ for native_query_key in ("url_params", "extra_form_data", "extra_filters"):
+ validation_data.pop(native_query_key, None)
+ for preserved_key in _BULLET_UNMODELED_PRESERVED_KEYS:
+ validation_data.pop(preserved_key, None)
+ if not validation_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT):
+ # A null marker records an explicit subject clear with no binding to
+ # validate; it is not an authored physical column.
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ if update_config is None or isinstance(update_config, BulletChartConfig):
+ if update_config is None or update_config.dimensions is None:
+ validation_data.pop("groupby", None)
Review Comment:
Good catch, confirmed: the validation copy dropped the inherited `groupby`, so `label: "year"` over a saved `["year"]` hierarchy passed. Fixed in https://github.com/apache/superset/commit/eade9f40c9b76661a7d98b7ac046518f4debc0bc — `validate_merged_bullet_form_data` now attaches the inherited hierarchy and re-runs `validate_roles_and_outputs`, which checks the metric output label against inherited dimension output labels (physical names and SQL dimension labels). Regression tests cover both the persisted and preview update paths for a physical and an adhoc SQL inherited dimension, plus a non-colliding label that still validates.
##########
superset/mcp_service/chart/validation/pipeline.py:
##########
@@ -157,6 +156,34 @@ def validate_request_with_warnings(
error_code="GANTT_SEMANTIC_VALIDATION_ERROR",
),
)
+ except ValueError as ex:
Review Comment:
Agreed. Fixed in https://github.com/apache/superset/commit/eade9f40c9b76661a7d98b7ac046518f4debc0bc: added `test_validation_pipeline_reports_ambiguous_bullet_reference_error_code`, which goes through `validate_request_with_warnings` with `Region`/`region` and a `rEgIoN` dimension and asserts `is_valid is False` and `error_code == "AMBIGUOUS_COLUMN_REFERENCE"` (verified it fails if the `except ValueError` handler is removed). The existing direct test is renamed to `test_bullet_normalization_raises_named_ambiguity_error` so its name matches what it exercises.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3924022507
##########
tests/unit_tests/mcp_service/chart/tool/test_get_chart_preview.py:
##########
@@ -45,12 +53,1126 @@
_first_query_has_fields,
_no_query_fields_error,
ASCIIPreviewStrategy,
+ get_chart_preview,
PreviewFormatStrategy,
TablePreviewStrategy,
)
from superset.utils import json as utils_json
+def _query_context_stub(form_data: dict[str, Any] | None = None) -> Any:
+ """Return the minimal real-shaped context needed by Jinja form-data seeding."""
+ return SimpleNamespace(form_data=form_data or {}, queries=[])
+
+
+def _entrypoint_preview(content: str) -> ChartPreview:
+ return ChartPreview(
+ chart_id=1,
+ chart_name="",
+ chart_type="bullet",
+ explore_url="",
+ content=ASCIIPreview(ascii_content=content, width=80, height=20),
+ chart_description="",
+ accessibility=AccessibilityMetadata(
+ color_blind_safe=True,
+ alt_text="",
+ high_contrast_available=False,
+ ),
+ performance=PerformanceMetadata(
+ query_duration_ms=0,
+ cache_status="miss",
+ optimization_suggestions=[],
+ ),
+ )
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ ("request_payload", "extra", "expected_type"),
+ [
+ ({"id": 1, "format": "ascii"}, 0, ChartPreview),
+ ({"form_data_key": "cached-preview", "format": "ascii"}, 1, ChartError),
+ ],
+ ids=["identifier-alias-exact-limit", "cached-preview-limit-plus-one"],
+)
+async def test_get_chart_preview_entrypoint_preflights_complete_exact_wire_response(
+ request_payload: dict[str, object], extra: int, expected_type: type[object]
+) -> None:
+ empty = _entrypoint_preview("")
+ filler = "x" * (
+ MAX_QUERY_RESULT_VALUE_BYTES - len(empty.model_dump_json().encode()) + extra
+ )
+ candidate = _entrypoint_preview(filler)
+ request = GetChartPreviewRequest.model_validate(request_payload)
+ ctx = MagicMock()
+ ctx.info = AsyncMock()
+ ctx.debug = AsyncMock()
+ ctx.warning = AsyncMock()
+
+ user = MagicMock(id=1, username="admin", roles=[], groups=[])
+ with (
+ patch("superset.mcp_service.auth.get_user_from_request", return_value=user),
+ patch(
+ "superset.mcp_service.chart.tool.get_chart_preview."
+ "_get_chart_preview_internal",
+ new=AsyncMock(return_value=candidate),
+ ),
+ ):
+ result = await get_chart_preview(request, ctx=ctx)
+
+ assert isinstance(result, expected_type)
+ if extra == 0:
+ assert len(candidate.model_dump_json().encode()) == (
+ MAX_QUERY_RESULT_VALUE_BYTES
+ )
+ assert result is candidate
+ else:
+ assert isinstance(result, ChartError)
+ assert result.error_type == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("format_", ["ascii", "vega_lite"])
+async def test_bullet_numeric_and_temporal_categories_reach_real_mcp_entrypoint(
+ format_: str,
+) -> None:
+ from contextlib import nullcontext
+
+ from fastmcp import Client
+
+ from superset.mcp_service.app import mcp
+
+ preview_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_preview"
+ )
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=121,
+ slice_name="Number boundaries",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ params=utils_json.dumps(form_data),
+ )
+ rows = [
+ {"Category": 9007199254740993, "Revenue": 1},
+ {"Category": Decimal("1.0000000000000001"), "Revenue": 2},
+ {"Category": Decimal("1.7976931348623159e308"), "Revenue": 3},
+ {"Category": date(2026, 9, 2), "Revenue": 4},
+ {
+ "Category": datetime(2026, 9, 2, 3, 4, 5, tzinfo=timezone.utc),
+ "Revenue": 5,
+ },
+ {
+ "Category": datetime(
+ 2023,
+ 11,
+ 5,
+ 1,
+ 30,
+ tzinfo=ZoneInfo("America/New_York"),
+ fold=0,
+ ),
+ "Revenue": 6,
+ },
+ {
+ "Category": datetime(
+ 2023,
+ 11,
+ 5,
+ 1,
+ 30,
+ tzinfo=ZoneInfo("America/New_York"),
+ fold=1,
+ ),
+ "Revenue": 7,
+ },
+ ]
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ }
+ ]
+ }
+
+ query_context = SimpleNamespace(
+ form_data={},
+ queries=[SimpleNamespace(metrics=["Revenue"], columns=["Category"])],
+ )
+ user = MagicMock(id=1, username="admin", roles=[], groups=[])
+ with (
+ patch("superset.mcp_service.auth.get_user_from_request", return_value=user),
+ patch("superset.mcp_service.auth.check_tool_permission", return_value=True),
+ patch.object(preview_module, "find_chart_by_identifier", return_value=chart),
+ patch.object(preview_module.db.session, "refresh", return_value=None),
+ patch.object(
+ preview_module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch.object(
+ preview_module.event_logger,
+ "log_context",
+ side_effect=lambda **_kwargs: nullcontext(),
+ ),
+ patch.object(
+ preview_module,
+ "build_query_context_from_form_data",
+ return_value=query_context,
+ ),
+ patch.object(preview_module, "set_query_context_form_data", return_value=None),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ patch.object(
+ preview_module, "get_superset_base_url", return_value="http://localhost"
+ ),
+ ):
+ async with Client(mcp) as client:
+ result = await client.call_tool(
+ "get_chart_preview",
+ {"request": {"id": 121, "format": format_}},
+ )
+
+ payload = utils_json.loads(result.content[0].text)
+ if format_ == "ascii":
+ content = payload["content"]["ascii_content"]
+ assert "9007199254740992" in content
+ assert "Infinity" in content
+ assert "1788307200000" in content
+ assert "1699162200000" in content
+ assert "1699165800000" in content
+ else:
+ specification = payload["content"]["specification"]
+ bar = next(
+ layer for layer in specification["layer"] if layer["mark"]["type"] == "bar"
+ )
+ category_field = bar["encoding"]["y"]["field"]
+ assert [row[category_field] for row in specification["data"]["values"]] == [
+ "9007199254740992",
+ "1",
+ "Infinity",
+ "1788307200000",
+ "1788318245000",
+ "1699162200000",
+ "1699165800000",
+ ]
+ assert [row["Category"] for row in specification["data"]["values"][3:]] == [
+ 1788307200000.0,
+ 1788318245000.0,
+ 1699162200000.0,
+ 1699165800000.0,
+ ]
+ assert bar["encoding"]["tooltip"][0]["field"] == category_field
+ assert "transform" not in specification
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("format_", ["ascii", "vega_lite"])
+async def test_bullet_timestamp_categories_from_dataframe_reach_fastmcp(
+ format_: str,
+) -> None:
+ from contextlib import nullcontext
+
+ from fastmcp import Client
+
+ from superset.dataframe import df_to_records
+ from superset.mcp_service.app import mcp
+
+ preview_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_preview"
+ )
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ zoneinfo_tz = ZoneInfo("America/New_York")
+ pytz_tz = pytz.timezone("America/New_York")
+ dateutil_dublin = dateutil_tz.gettz("Europe/Dublin")
+ dateutil_new_york = dateutil_tz.gettz("America/New_York")
+ assert dateutil_dublin is not None
+ assert dateutil_new_york is not None
+ dublin_fold = datetime(
+ 2024,
+ 10,
+ 27,
+ 1,
+ 30,
+ 0,
+ 123456,
+ tzinfo=dateutil_dublin,
+ fold=1,
+ )
+ new_york_gap = datetime(2024, 3, 10, 2, 30, 0, 123456, tzinfo=dateutil_new_york)
+ source_values = [
+ pd.Timestamp("2024-01-02 03:04:05.123456789"),
+ pd.Timestamp("2024-01-02 08:34:05.123456789+05:30"),
+ pd.Timestamp(datetime(2024, 11, 3, 1, 30, tzinfo=zoneinfo_tz, fold=0)),
+ pd.Timestamp(datetime(2024, 11, 3, 1, 30, tzinfo=zoneinfo_tz, fold=1)),
+ pd.Timestamp(pytz_tz.localize(datetime(2024, 11, 3, 1, 30), is_dst=True)),
+ pd.Timestamp(pytz_tz.localize(datetime(2024, 11, 3, 1, 30), is_dst=False)),
+ pd.Timestamp("1969-12-31 23:59:59.999999999"),
+ date(2024, 1, 2),
+ pd.NaT,
+ dublin_fold,
+ new_york_gap,
+ datetime(2040, 7, 1, 12, 0, 0, 123456, tzinfo=dateutil_new_york),
+ datetime(
+ 2024,
+ 3,
+ 10,
+ 2,
+ 30,
+ 0,
+ 123456,
+ tzinfo=dateutil_tz.tzoffset("EDT", -4 * 3600),
+ ),
+ datetime(2024, 3, 10, 6, 30, 0, 123456, tzinfo=timezone.utc),
+ pd.Timestamp(dublin_fold),
+ pd.Timestamp(new_york_gap),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+ if format_ == "ascii":
+ # ASCII intentionally displays at most ten categories. Keep every
+ # dateutil named-zone edge in this public-format invocation.
+ rows = rows[9:12]
+ expected = [
+ "1704164645123.456",
+ "1704164645123.456",
+ "1730611800000",
+ "1730615400000",
+ "1730611800000",
+ "1730615400000",
+ "-0.0010000000000287557",
+ "1704153600000",
+ "null",
+ "1729989000123.456",
+ "1710052200123.456",
+ "2224774800123.456",
+ "1710052200123.456",
+ "1710052200123.456",
+ "1729989000123.456",
+ "1710052200123.456",
+ ]
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=122,
+ slice_name="Timestamp categories",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ params=utils_json.dumps(form_data),
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ }
+ ]
+ }
+
+ query_context = SimpleNamespace(
+ form_data={},
+ queries=[SimpleNamespace(metrics=["Revenue"], columns=["Category"])],
+ )
+ user = MagicMock(id=1, username="admin", roles=[], groups=[])
+ with (
+ patch("superset.mcp_service.auth.get_user_from_request", return_value=user),
+ patch("superset.mcp_service.auth.check_tool_permission", return_value=True),
+ patch.object(preview_module, "find_chart_by_identifier", return_value=chart),
+ patch.object(preview_module.db.session, "refresh", return_value=None),
+ patch.object(
+ preview_module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch.object(
+ preview_module.event_logger,
+ "log_context",
+ side_effect=lambda **_kwargs: nullcontext(),
+ ),
+ patch.object(
+ preview_module,
+ "build_query_context_from_form_data",
+ return_value=query_context,
+ ),
+ patch.object(preview_module, "set_query_context_form_data", return_value=None),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ patch.object(
+ preview_module, "get_superset_base_url", return_value="http://localhost"
+ ),
+ ):
+ async with Client(mcp) as client:
+ result = await client.call_tool(
+ "get_chart_preview",
+ {"request": {"id": 122, "format": format_}},
+ )
+
+ payload = utils_json.loads(result.content[0].text)
+ if format_ == "ascii":
+ content = payload["content"]["ascii_content"]
+ for category in set(expected[9:12]):
+ assert category[:20] in content
+ else:
+ specification = payload["content"]["specification"]
+ bar = next(
+ layer for layer in specification["layer"] if layer["mark"]["type"] == "bar"
+ )
+ category_field = bar["encoding"]["y"]["field"]
+ assert [row[category_field] for row in specification["data"]["values"]] == (
+ expected
+ )
+ assert bar["encoding"]["tooltip"][0]["field"] == category_field
+ assert [row["Category"] for row in specification["data"]["values"]] == [
+ 1704164645123.456,
+ 1704164645123.456,
+ 1730611800000.0,
+ 1730615400000.0,
+ 1730611800000.0,
+ 1730615400000.0,
+ -0.0010000000000287557,
+ 1704153600000.0,
+ None,
+ 1729989000123.456,
+ 1710052200123.456,
+ 2224774800123.456,
+ 1710052200123.456,
+ 1710052200123.456,
+ 1729989000123.456,
+ 1710052200123.456,
+ ]
+
+
+@pytest.mark.asyncio
+async def test_transitionless_dateutil_dataframe_reaches_fastmcp_preview() -> None:
+ from contextlib import nullcontext
+
+ from fastmcp import Client
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+ from superset.mcp_service.app import mcp
+ from superset.mcp_service.chart.preview_utils import _javascript_number_string
+ from superset.utils.json import json_int_dttm_ser
+
+ preview_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_preview"
+ )
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ names = [
+ "UTC",
+ "GMT",
+ "Universal",
+ "Zulu",
+ "EST",
+ "HST",
+ "MST",
+ "Etc/GMT+1",
+ "Etc/GMT-2",
+ ]
+ values = []
+ for getter in (dateutil_tz.gettz, get_zonefile_instance().get):
+ for name in names:
+ timezone_value = getter(name)
+ assert timezone_value is not None
+ values.append(
+ datetime(2040, 7, 1, 12, 34, 56, 123456, tzinfo=timezone_value)
+ )
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(values, dtype=object),
+ "Revenue": range(1, len(values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+
+ class _ProducerContext:
+ result_type = ChartDataResultType.FULL
+
+ def get_payload(self, **_kwargs: Any) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ producer_result = ProducerChartDataCommand(
+ _ProducerContext() # type: ignore[arg-type]
+ ).run()
+ expected_numbers = [json_int_dttm_ser(value) for value in values]
+ expected_categories = [
+ _javascript_number_string(float(value)) for value in expected_numbers
+ ]
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=123,
+ slice_name="Transitionless timestamps",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ params=utils_json.dumps(form_data),
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return producer_result
+
+ query_context = SimpleNamespace(
+ form_data={},
+ queries=[SimpleNamespace(metrics=["Revenue"], columns=["Category"])],
+ )
+ user = MagicMock(id=1, username="admin", roles=[], groups=[])
+ with (
+ patch("superset.mcp_service.auth.get_user_from_request", return_value=user),
+ patch("superset.mcp_service.auth.check_tool_permission", return_value=True),
+ patch.object(preview_module, "find_chart_by_identifier", return_value=chart),
+ patch.object(preview_module.db.session, "refresh", return_value=None),
+ patch.object(
+ preview_module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch.object(
+ preview_module.event_logger,
+ "log_context",
+ side_effect=lambda **_kwargs: nullcontext(),
+ ),
+ patch.object(
+ preview_module,
+ "build_query_context_from_form_data",
+ return_value=query_context,
+ ),
+ patch.object(preview_module, "set_query_context_form_data", return_value=None),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ patch.object(
+ preview_module, "get_superset_base_url", return_value="http://localhost"
+ ),
+ ):
+ async with Client(mcp) as client:
+ result = await client.call_tool(
+ "get_chart_preview",
+ {"request": {"id": 123, "format": "vega_lite"}},
+ )
+
+ payload = utils_json.loads(result.content[0].text)
+ specification = payload["content"]["specification"]
+ bar = next(
+ layer for layer in specification["layer"] if layer["mark"]["type"] == "bar"
+ )
+ category_field = bar["encoding"]["y"]["field"]
+ assert [row["Category"] for row in specification["data"]["values"]] == (
+ expected_numbers
+ )
+ assert [row[category_field] for row in specification["data"]["values"]] == (
+ expected_categories
+ )
+ assert bar["encoding"]["tooltip"][0]["field"] == category_field
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("format_", ["ascii", "vega_lite"])
+async def test_duration_dataframe_reaches_fastmcp_bullet_preview(
+ format_: str,
+) -> None:
+ from contextlib import nullcontext
+
+ import numpy as np
+ from fastmcp import Client
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+ from superset.mcp_service.app import mcp
+
+ preview_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_preview"
+ )
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ source_values = [
+ timedelta(0),
+ timedelta(days=1, seconds=2, microseconds=3),
+ pd.Timedelta(-1, unit="ns"),
+ np.timedelta64(123456789, "ns"),
+ np.timedelta64("NaT"),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Duration": pd.Series(source_values, dtype=object),
+ "Revenue": [50, 120, 350, 0, 200],
+ }
+ ),
+ convert_big_integers=False,
+ )
+ expected = [
+ "null"
+ if row["Duration"] is None
+ else utils_json.loads(
+ utils_json.dumps(row["Duration"], default=utils_json.json_int_dttm_ser)
+ )
+ for row in rows
+ ]
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Test crashes on np.timedelta64</b></div>
<div id="fix">
The `expected` comprehension calls `utils_json.dumps(row["Duration"], default=json_int_dttm_ser)`, which falls through to `base_json_conv`. That function handles `timedelta`/`pd.Timedelta` but not `np.timedelta64` (not a `timedelta` subclass), so it raises `TypeError` for the two `np.timedelta64` source values and the test crashes before the tool call. Also `df_to_records` leaves `np.timedelta64("NaT")` as-is (not None), so the `"null"` branch never fires for NaT. Normalize `np.timedelta64` (NaT→None, else `pd.Timedelta`) to mirror `_chart_data_duration_text`.
</div>
</div>
<small><i>Code Review Run #d53793</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
##########
tests/unit_tests/mcp_service/chart/test_bullet_chart.py:
##########
@@ -0,0 +1,3380 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Product-path coverage for typed ECharts Bullet MCP support."""
+
+import math
+from datetime import date, datetime, time, timedelta, timezone, tzinfo
+from decimal import Decimal
+from enum import Enum, IntEnum, StrEnum
+from types import SimpleNamespace
+from typing import Any
+from unittest.mock import AsyncMock, MagicMock, patch
+from uuid import UUID
+from zoneinfo import ZoneInfo
+
+import numpy as np
+import pandas as pd
+import pytest
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.zoneinfo import get_zonefile_instance
+from pydantic import TypeAdapter, ValidationError
+
+from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+)
+from superset.mcp_service.chart.chart_utils import (
+ analyze_chart_capabilities,
+ map_bullet_config,
+ map_config_to_form_data,
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ merge_update_form_data,
+ validate_merged_bullet_form_data,
+)
+from superset.mcp_service.chart.compile import _compile_chart
+from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_preview_from_data,
+ _generate_vega_lite_preview_from_data,
+ _javascript_number_string,
+ BulletOutputError,
+ generate_preview_from_form_data,
+ resolve_bullet_render_model,
+)
+from superset.mcp_service.chart.query_result import (
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+)
+from superset.mcp_service.chart.schemas import (
+ ASCIIPreview,
+ BulletChartConfig,
+ ChartConfig,
+ ChartError,
+ DataColumn,
+ GenerateChartRequest,
+ GetChartPreviewRequest,
+ UpdateChartPreviewRequest,
+ UpdateChartRequest,
+ VegaLitePreview,
+ XYChartConfig,
+)
+from superset.mcp_service.chart.tool.generate_chart import generate_chart
+from superset.mcp_service.chart.tool.get_chart_data import (
+ _candidates_single_numeric,
+ _VIZ_CATEGORY,
+)
+from superset.mcp_service.chart.tool.get_chart_preview import (
+ ASCIIPreviewStrategy,
+ TablePreviewStrategy,
+ VegaLitePreviewStrategy,
+)
+from superset.mcp_service.chart.tool.get_chart_type_schema import (
+ _get_chart_type_schema_impl,
+ VALID_CHART_TYPES,
+)
+from superset.mcp_service.chart.tool.update_chart import (
+ _build_preview_form_data,
+ _build_update_payload,
+ update_chart,
+)
+from superset.mcp_service.chart.tool.update_chart_preview import update_chart_preview
+from superset.mcp_service.chart.validation.dataset_validator import DatasetValidator
+from superset.mcp_service.common.error_schemas import DatasetContext
+from superset.utils.json import json_int_dttm_ser
+
+
+def _reject_scalar_conversion(*_args: object, **_kwargs: object) -> Any:
+ raise AssertionError("hostile query scalar method must not run")
+
+
+class _PathHostileStr(str):
+ __getitem__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _PathHostileEnum(str, Enum):
+ FAILED = "warehouse unavailable"
+
+ @property
+ def value(self) -> str:
+ """Reject the public descriptor while preserving Enum's stored value."""
+ return _reject_scalar_conversion()
+
+ __getitem__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileInt(int):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileFloat(float):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileStr(str):
+ __str__ = _reject_scalar_conversion
+ strip = _reject_scalar_conversion
+
+
+class _OutputHostileDecimal(Decimal):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputSafeIntEnum(IntEnum):
+ VALUE = 12
+
+
+class _OutputSafeStrEnum(StrEnum):
+ VALUE = "12.5"
+
+
+_OutputSafeIntEnum.__float__ = _reject_scalar_conversion # type: ignore[method-assign]
+_OutputSafeIntEnum.__str__ = _reject_scalar_conversion # type: ignore[method-assign]
+_OutputSafeStrEnum.__float__ = _reject_scalar_conversion # type: ignore[attr-defined]
+_OutputSafeStrEnum.__str__ = _reject_scalar_conversion # type: ignore[method-assign]
+
+
+def _simple_metric(name: str = "revenue") -> dict[str, str]:
+ return {"name": name, "aggregate": "SUM"}
+
+
+def _tool_user() -> SimpleNamespace:
+ return SimpleNamespace(id=1, username="admin", roles=[], groups=[])
+
+
+def _orm_dataset() -> SimpleNamespace:
+ def column(
+ name: str, type_: str, *, temporal: bool = False, numeric: bool = False
+ ) -> SimpleNamespace:
+ return SimpleNamespace(
+ column_name=name,
+ type=type_,
+ is_temporal=temporal,
+ is_numeric=numeric,
+ is_dttm=temporal,
+ python_date_format=None,
+ )
+
+ return SimpleNamespace(
+ id=7,
+ table_name="sales",
+ schema=None,
+ main_dttm_col="OrderDate",
+ database=SimpleNamespace(database_name="main", db_engine_spec=None),
+ columns=[
+ column("Revenue", "NUMERIC", numeric=True),
+ column("Region", "VARCHAR"),
+ column("Team", "VARCHAR"),
+ column("Status", "VARCHAR"),
+ column("OrderDate", "TIMESTAMP", temporal=True),
+ column("EventDate", "TIMESTAMP", temporal=True),
+ ],
+ metrics=[
+ SimpleNamespace(
+ metric_name="SavedRevenue",
+ expression="SUM(Revenue)",
+ description=None,
+ )
+ ],
+ )
+
+
+def test_bullet_discriminated_union_uses_exact_tag() -> None:
+ config = TypeAdapter(ChartConfig).validate_python(
+ {"chart_type": "bullet", "metric": _simple_metric()}
+ )
+ assert isinstance(config, BulletChartConfig)
+ with pytest.raises(ValidationError):
+ TypeAdapter(ChartConfig).validate_python(
+ {"chart_type": "bullet_chart", "metric": _simple_metric()}
+ )
+
+
+def test_bullet_equal_dimension_aliases_are_order_independent_and_round_trip() -> None:
+ for payload in (
+ {
+ "dimensions": [{"name": "Region", "label": "Market"}, "Team"],
+ "groupby": ["Region", {"column_name": "Team"}],
+ },
+ {
+ "groupby": ["Region", {"column": "Team"}],
+ "dimensions": [{"name": "Region", "label": "Market"}, "Team"],
+ },
+ ):
+ config = BulletChartConfig.model_validate(
+ {"metric": _simple_metric(), **payload}
+ )
+ assert [dimension.name for dimension in config.dimensions or []] == [
+ "Region",
+ "Team",
+ ]
+ mapped = map_bullet_config(config)
+ assert mapped["groupby"] == ["Region", "Team"]
+ round_trip = BulletChartConfig.model_validate(mapped)
+ assert [dimension.name for dimension in round_trip.dimensions or []] == [
+ "Region",
+ "Team",
+ ]
+
+
+@pytest.mark.parametrize(
+ "request_payload",
+ [
+ {"dataset_id": 7},
+ {"identifier": 9},
+ {"dataset_id": 7, "form_data_key": "preview"},
+ ],
+)
+@pytest.mark.parametrize("reverse", [False, True])
+def test_bullet_request_models_reject_conflicting_dimension_aliases(
+ request_payload: dict[str, object], reverse: bool
+) -> None:
+ aliases = [
+ ("dimensions", [{"name": "Region"}, {"name": "Team"}]),
+ ("groupby", ["Team", "Region"]),
+ ]
+ if reverse:
+ aliases.reverse()
+ config = {"chart_type": "bullet", "metric": _simple_metric(), **dict(aliases)}
+ payload = {**request_payload, "config": config}
+ request_type = (
+ UpdateChartRequest
+ if "identifier" in request_payload
+ else (
+ UpdateChartPreviewRequest
+ if "form_data_key" in request_payload
+ else GenerateChartRequest
+ )
+ )
+ with pytest.raises(ValidationError, match="Conflicting Bullet dimension aliases"):
+ request_type.model_validate(payload)
+
+
+@pytest.mark.parametrize(
+ "metric",
+ [
+ {"name": "revenue", "aggregate": "SUM", "label": "Revenue"},
+ {"name": "saved_revenue", "saved_metric": True},
+ {"sql_expression": "SUM(revenue) / COUNT(*)", "label": "Average"},
+ ],
+)
+def test_bullet_accepts_simple_saved_and_sql_metrics(metric: dict[str, object]) -> None:
+ config = BulletChartConfig(metric=metric)
+ form_data = map_bullet_config(config)
+ assert form_data["viz_type"] == "bullet"
+ assert form_data["metric"]
+
+
+def test_bullet_native_form_data_round_trip_is_semantically_stable() -> None:
+ native = {
+ "viz_type": "bullet",
+ "datasource": "7__table",
+ "metric": {
+ "aggregate": "SUM",
+ "column": {"column_name": "Revenue"},
+ "expressionType": "SIMPLE",
+ "label": "Total Revenue",
+ },
+ "groupby": ["Region", "Team"],
+ "ranges": "100,250,500",
+ "range_labels": "Minimum,Target,Stretch",
+ "markers": "300",
+ "marker_labels": "Plan",
+ "marker_lines": "400",
+ "marker_line_labels": "Forecast",
+ "y_axis_format": "$,.0f",
+ "show_labels": False,
+ "show_legend": True,
+ "row_limit": 250,
+ "orderby": [["Region", True], ["Total Revenue", False]],
+ "adhoc_filters": [
+ {
+ "clause": "WHERE",
+ "expressionType": "SIMPLE",
+ "subject": "Status",
+ "operator": "==",
+ "comparator": "Active",
+ }
+ ],
+ }
+ config = BulletChartConfig.model_validate(native)
+ mapped = map_bullet_config(config)
+
+ assert mapped["metric"]["label"] == "Total Revenue"
+ assert mapped["groupby"] == ["Region", "Team"]
+ assert mapped["ranges"] == "100,250,500"
+ assert mapped["range_labels"] == "Minimum,Target,Stretch"
+ assert mapped["markers"] == "300"
+ assert mapped["marker_lines"] == "400"
+ assert mapped["show_labels"] is False
+ assert mapped["show_legend"] is True
+ assert mapped["orderby"][0] == ["Region", True]
+ assert mapped["orderby"][1][0]["label"] == "Total Revenue"
+ assert mapped["adhoc_filters"][0]["subject"] == "Status"
+
+
+def test_bullet_presentation_numbers_use_shortest_round_trip_safe_tokens() -> None:
+ ranges = [1.2345678901234567, 1.7976931348623157e308]
+ markers = [5e-324, -0.0]
+ marker_lines = [9.876543210987654e-200]
+ config = BulletChartConfig(
+ metric=_simple_metric(),
+ ranges=ranges,
+ markers=markers,
+ marker_lines=marker_lines,
+ show_legend=True,
+ )
+ mapped = map_bullet_config(config)
+
+ for key, expected in (
+ ("ranges", ranges),
+ ("markers", markers),
+ ("marker_lines", marker_lines),
+ ):
+ tokens = mapped[key].split(",")
+ assert [float(token) for token in tokens] == expected
+ assert all(
+ float(token).hex() == value.hex()
+ for token, value in zip(tokens, expected, strict=True)
+ )
+
+ round_trip = BulletChartConfig.model_validate(mapped)
+ assert round_trip.ranges == ranges
+ assert round_trip.markers == markers
+ assert round_trip.marker_lines == marker_lines
+
+ model = resolve_bullet_render_model(
+ [{"SUM(revenue)": 1.0}],
+ mapped,
+ )
+ assert model.ranges == ranges
+ assert model.markers == markers
+ assert model.marker_lines == marker_lines
+ assert (
+ "1.7976931348623157e+308"
+ in _generate_ascii_preview_from_data(
+ [{"SUM(revenue)": 1.0}], mapped
+ ).ascii_content
+ )
+ vega = _generate_vega_lite_preview_from_data([{"SUM(revenue)": 1.0}], mapped)
+ assert vega.specification["layer"]
+
+
+def test_bullet_native_saved_metric_and_legacy_metric_aliases() -> None:
+ saved = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": "saved_revenue"}
+ )
+ legacy = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": "sum__revenue"}
+ )
+ assert saved.metric.saved_metric is True
+ assert saved.metric.name == "saved_revenue"
+ assert legacy.metric.name == "sum__revenue"
+ assert legacy.metric.saved_metric is True
+
+
+@pytest.mark.parametrize("metric_name", ["sum__num", "sum__SP_POP_TOTL"])
+@pytest.mark.parametrize(
+ ("request_type", "request_fields"),
+ [
+ (GenerateChartRequest, {"dataset_id": 7}),
+ (UpdateChartRequest, {"identifier": 9}),
+ (UpdateChartPreviewRequest, {"dataset_id": 7}),
+ ],
+)
+def test_bullet_repository_metric_names_round_trip_as_saved_metrics_on_all_requests(
+ metric_name: str,
+ request_type: type[
+ GenerateChartRequest | UpdateChartRequest | UpdateChartPreviewRequest
+ ],
+ request_fields: dict[str, object],
+) -> None:
+ request = request_type.model_validate(
+ {**request_fields, "config": {"chart_type": "bullet", "metric": metric_name}}
+ )
+ config = request.config
+ assert isinstance(config, BulletChartConfig)
+ assert config.metric.saved_metric is True
+ assert map_bullet_config(config)["metric"] == metric_name
+
+
+def test_bullet_legacy_label_only_saved_metric_adapter_is_strict_and_bounded() -> None:
+ config = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": {"label": "sum__num"}}
+ )
+ assert config.metric.saved_metric is True
+ assert map_bullet_config(config)["metric"] == "sum__num"
+
+ with pytest.raises(ValidationError):
+ BulletChartConfig.model_validate(
+ {
+ "viz_type": "bullet",
+ "metric": {"label": "sum__num", "aggregate": "SUM"},
+ }
+ )
+ with pytest.raises(ValidationError, match="at most 255"):
+ BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": {"label": "m" * 256}}
+ )
+
+
+@pytest.mark.parametrize(
+ "metric",
+ [
+ "SavedRevenue",
+ {
+ "aggregate": "SUM",
+ "column": {"column_name": "Revenue"},
+ "expressionType": "SIMPLE",
+ "label": "Simple Revenue",
+ },
+ {
+ "aggregate": None,
+ "column": None,
+ "expressionType": "SQL",
+ "sqlExpression": "SUM(Revenue)",
+ "label": "SQL Revenue",
+ },
+ ],
+)
+def test_bullet_all_metric_shapes_round_trip_full_native_presentation(
+ metric: object,
+) -> None:
+ native = {
+ "viz_type": "bullet",
+ "metric": metric,
+ "groupby": ["Region"],
+ "ranges": "50,100",
+ "range_labels": "Low,High",
+ "markers": "75",
+ "marker_labels": "Plan",
+ "marker_lines": "90",
+ "marker_line_labels": "Forecast",
+ "y_axis_format": "$,.0f",
+ "show_labels": True,
+ "show_legend": True,
+ }
+ mapped = map_bullet_config(BulletChartConfig.model_validate(native))
+ assert validate_merged_bullet_form_data(mapped) is not None
+ assert mapped["groupby"] == ["Region"]
+ assert mapped["ranges"] == "50,100"
+ assert mapped["marker_line_labels"] == "Forecast"
+
+
+def test_bullet_rejects_invalid_roles_and_output_collisions() -> None:
+ with pytest.raises(ValidationError, match="physical dimension"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[{"name": "region", "aggregate": "COUNT"}],
+ )
+ with pytest.raises(ValidationError, match="Duplicate Bullet dimension"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[{"name": "Region"}, {"name": "region"}],
+ )
+ with pytest.raises(ValidationError, match="conflicts with a dimension"):
+ BulletChartConfig(
+ metric={"name": "revenue", "aggregate": "SUM", "label": "Region"},
+ dimensions=[{"name": "region", "label": "Region"}],
+ )
+
+
+def test_bullet_rejects_misaligned_labels_and_bad_order_target() -> None:
+ with pytest.raises(ValidationError, match="one label per ranges"):
+ BulletChartConfig(
+ metric=_simple_metric(), ranges=[1, 2], range_labels=["Only one"]
+ )
+ with pytest.raises(ValidationError, match="unknown: not_a_role"):
+ BulletChartConfig(metric=_simple_metric(), order_by=[{"column": "not_a_role"}])
+
+
+def test_bullet_dimension_labels_are_input_aliases_not_result_aliases() -> None:
+ config = BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Total"},
+ dimensions=[
+ {"name": "Team", "label": "Region"},
+ {"name": "Region", "label": "Market"},
+ ],
+ # The exact physical Region must win over Team's display label.
+ order_by=[
+ {"column": "Region", "ascending": True},
+ {"column": "Revenue", "ascending": False},
+ ],
+ )
+ form_data = map_bullet_config(config)
+ assert form_data["groupby"] == ["Team", "Region"]
+ assert form_data["orderby"] == [
+ ["Region", True],
+ [form_data["metric"], False],
+ ]
+
+ label_order = map_bullet_config(
+ BulletChartConfig(
+ metric=config.metric,
+ dimensions=config.dimensions,
+ order_by=[{"column": "Market"}],
+ )
+ )
+ assert label_order["orderby"] == [["Region", False]]
+
+ model = resolve_bullet_render_model(
+ [{"Team": "Blue", "Region": "North", "Total": 10}], form_data
+ )
+ assert model.dimensions == ["Team", "Region"]
+ assert [model.rows[0][name] for name in model.dimensions] == ["Blue", "North"]
+
+
+def test_bullet_rejects_ambiguous_display_alias_for_ordering() -> None:
+ with pytest.raises(ValidationError, match="ambiguous display alias"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[
+ {"name": "Region", "label": "Area"},
+ {"name": "Team", "label": "area"},
+ ],
+ order_by=[{"column": "AREA"}],
+ )
+
+
+@pytest.mark.parametrize(
+ ("metric", "order_target", "output"),
+ [
+ (
+ {"name": "SavedRevenue", "saved_metric": True, "label": "Friendly"},
+ "Friendly",
+ "SavedRevenue",
+ ),
+ (
+ {"name": "Revenue", "aggregate": "SUM", "label": "Simple Total"},
+ "Revenue",
+ "Simple Total",
+ ),
+ (
+ {"sql_expression": "SUM(Revenue)", "label": "SQL Total"},
+ "SQL Total",
+ "SQL Total",
+ ),
+ ],
+)
+def test_bullet_metric_shapes_share_physical_dimension_output_contract(
+ metric: dict[str, object], order_target: str, output: str
+) -> None:
+ config = BulletChartConfig(
+ metric=metric,
+ dimensions=[{"name": "Region", "label": "Market"}],
+ order_by=[{"column": order_target}],
+ )
+ form_data = map_bullet_config(config)
+ assert form_data["groupby"] == ["Region"]
+ assert form_data["orderby"] == [[form_data["metric"], False]]
+ model = resolve_bullet_render_model([{"Region": "North", output: 12}], form_data)
+ assert model.metric_field == output
+ assert model.dimensions == ["Region"]
+
+
+def test_bullet_mapper_preserves_omission_and_honors_explicit_values() -> None:
+ omitted = map_bullet_config(BulletChartConfig(metric=_simple_metric()))
+ explicit = map_bullet_config(
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[],
+ filters=[],
+ ranges=[],
+ show_labels=False,
+ show_legend=False,
+ row_limit=42,
+ time_range=None,
+ )
+ )
+ for key in (
+ "groupby",
+ "adhoc_filters",
+ "ranges",
+ "show_labels",
+ "show_legend",
+ "row_limit",
+ "time_range",
+ ):
+ assert key not in omitted
+ assert explicit["groupby"] == []
+ assert explicit["adhoc_filters"] == []
+ assert explicit["ranges"] == ""
+ assert explicit["show_labels"] is False
+ assert explicit["show_legend"] is False
+ assert explicit["row_limit"] == 42
+ assert explicit["time_range"] is None
+
+
+def test_bullet_registry_schema_and_recommendation_metadata() -> None:
+ from superset.mcp_service.app import get_default_instructions
+ from superset.mcp_service.chart.registry import display_name_for_viz_type, get
+
+ plugin = get("bullet")
+ assert plugin is not None
+ assert plugin.resolve_viz_type(None) == "bullet"
+ assert display_name_for_viz_type("bullet") == "Bullet Chart"
+ assert "bullet" in VALID_CHART_TYPES
+ discovered = _get_chart_type_schema_impl("bullet")
+ assert discovered["chart_type"] == "bullet"
+ assert discovered["examples"][0]["ranges"] == [100000, 250000, 500000]
+ assert _VIZ_CATEGORY["bullet"] == "bullet"
+ candidates = _candidates_single_numeric(
+ DataColumn(
+ name="Revenue",
+ display_name="Revenue",
+ data_type="numeric",
+ sample_values=[1],
+ null_count=0,
+ unique_count=1,
+ ),
+ row_count=1,
+ )
+ assert "bullet chart" in candidates
+ guidance = get_default_instructions()
+ assert 'chart_type="bullet": Bullet Chart' in guidance
+ assert "waterfall, bullet, and interactive_pivot" in guidance
+
+
+def test_bullet_dataset_normalization_canonicalizes_every_reference() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "Region", "type": "VARCHAR"},
+ {"name": "OrderDate", "type": "TIMESTAMP", "is_temporal": True},
+ {"name": "Status", "type": "VARCHAR"},
+ ],
+ available_metrics=[],
+ )
+ config = BulletChartConfig(
+ metric={"name": "revenue", "aggregate": "SUM"},
+ dimensions=[{"name": "region"}],
+ temporal_column="orderdate",
+ filters=[{"column": "status", "op": "=", "value": "active"}],
+ order_by=[{"column": "region", "ascending": True}],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ normalized = plugin.normalize_column_refs(config, context)
+ assert normalized.metric.name == "Revenue"
+ assert normalized.dimensions[0].name == "Region"
+ assert normalized.temporal_column == "OrderDate"
+ assert normalized.filters[0].column == "Status"
+ assert normalized.order_by[0].column == "Region"
+ assert normalized.model_fields_set == config.model_fields_set
+
+
+def test_bullet_dataset_normalization_rejects_ambiguous_casefold_candidates() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "revenue", "type": "NUMERIC", "is_numeric": True},
+ ],
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ config = BulletChartConfig(metric={"name": "REVENUE", "aggregate": "SUM"})
+ with pytest.raises(ValueError, match="Revenue, revenue"):
+ plugin.normalize_column_refs(config, context)
+
+
+def test_bullet_numeric_output_constraint_rejects_text_min() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[{"name": "status", "type": "VARCHAR"}],
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ config = BulletChartConfig(metric={"name": "status", "aggregate": "MIN"})
+ with patch.object(DatasetValidator, "_get_dataset_context", return_value=context):
+ error = plugin.post_map_validate(config, {}, dataset_id=7)
+ assert error is not None
+ assert error.error_type == "non_numeric_bullet_metric"
+
+
+@pytest.mark.parametrize("reverse_metadata", [False, True])
+def test_bullet_exact_case_type_and_role_resolution_is_order_independent(
+ reverse_metadata: bool,
+) -> None:
+ from superset.mcp_service.chart.registry import get
+
+ columns = [
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "revenue", "type": "VARCHAR", "is_numeric": False},
+ {"name": "Region", "type": "VARCHAR"},
+ ]
+ if reverse_metadata:
+ columns.reverse()
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=columns,
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+
+ numeric = BulletChartConfig(metric={"name": "Revenue", "aggregate": "SUM"})
+ text = BulletChartConfig(metric={"name": "revenue", "aggregate": "MIN"})
+ with patch.object(DatasetValidator, "_get_dataset_context", return_value=context):
+ assert plugin.post_map_validate(numeric, {}, dataset_id=7) is None
+ error = plugin.post_map_validate(text, {}, dataset_id=7)
+ assert error is not None
+ assert error.error_type == "non_numeric_bullet_metric"
+
+ roles = BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM"},
+ dimensions=[{"name": "revenue"}, {"name": "Region"}],
+ filters=[{"column": "revenue", "op": "=", "value": "retail"}],
+ order_by=[{"column": "revenue", "ascending": True}],
+ )
+ normalized = plugin.normalize_column_refs(roles, context)
+ assert normalized.metric.name == "Revenue"
+ assert [dimension.name for dimension in normalized.dimensions or []] == [
+ "revenue",
+ "Region",
+ ]
+ assert normalized.filters
+ assert normalized.filters[0].column == "revenue"
+ assert normalized.order_by[0].column == "revenue"
+
+ ambiguous = BulletChartConfig(metric={"name": "REVENUE", "aggregate": "SUM"})
+ with pytest.raises(ValueError, match="Ambiguous"):
+ plugin.normalize_column_refs(ambiguous, context)
+
+
+@pytest.mark.parametrize("reverse_metadata", [False, True])
+def test_generic_aggregation_validation_uses_exact_case_before_type(
+ reverse_metadata: bool,
+) -> None:
+ from superset.mcp_service.chart.schemas import PieChartConfig
+
+ columns = [
+ {"name": "Amount", "type": "BIGINT", "is_numeric": True},
+ {"name": "amount", "type": "VARCHAR", "is_numeric": False},
+ ]
+ if reverse_metadata:
+ columns.reverse()
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=columns,
+ available_metrics=[],
+ )
+
+ assert (
+ DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "Amount", "aggregate": "SUM"}).metric],
+ context,
+ )
+ == []
+ )
+ errors = DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "amount", "aggregate": "SUM"}).metric],
+ context,
+ )
+ assert errors
+ assert errors[0].error_type == "invalid_aggregation"
+
+ ambiguous = DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "AMOUNT", "aggregate": "SUM"}).metric],
+ context,
+ )
+ assert ambiguous
+ assert ambiguous[0].error_type == "ambiguous_column_reference"
+
+ valid, error = DatasetValidator.validate_against_dataset(
+ PieChartConfig(
+ dimension={"name": "amount"},
+ metric={"name": "Amount", "aggregate": "SUM"},
+ ),
+ 7,
+ dataset_context=context,
+ )
+ assert valid is True
+ assert error is None
+
+
+@pytest.mark.parametrize(
+ ("metric", "field"),
+ [
+ ({"name": "Revenue", "aggregate": "SUM", "label": "Simple"}, "Simple"),
+ ({"name": "SavedRevenue", "saved_metric": True}, "SavedRevenue"),
+ ({"sql_expression": "SUM(Revenue)", "label": "SQL Total"}, "SQL Total"),
+ ],
+)
+def test_bullet_result_roles_are_exact_for_every_metric_shape(
+ metric: dict[str, object], field: str
+) -> None:
+ form_data = map_bullet_config(BulletChartConfig(metric=metric))
+ model = resolve_bullet_render_model([{field.swapcase(): "12.5"}], form_data)
+ assert model.metric_field == field.swapcase()
+ assert model.measures == [12.5]
+
+
+@pytest.mark.parametrize(
+ "rows, message",
+ [
+ ([{"other": 123}], "missing"),
+ ([{"Revenue": "not a number"}], "non-numeric text"),
+ ([{"Revenue": math.nan}], "NaN or infinite"),
+ ([{"Revenue": math.inf}], "NaN or infinite"),
+ ([{"Revenue": 1}, {}], "row 1.*missing"),
+ ([{"REVENUE": 1, "revenue": 2}], "ambiguous"),
+ ],
+)
+def test_bullet_result_validation_rejects_malformed_rows(
+ rows: list[dict[str, object]], message: str
+) -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "amount", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ with pytest.raises(BulletOutputError, match=message):
+ resolve_bullet_render_model(rows, form_data)
+
+
+def test_bullet_result_validation_accepts_null_and_numeric_strings() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "amount", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Region"}],
+ )
+ )
+ model = resolve_bullet_render_model(
+ [
+ {"Region": "North", "Revenue": None},
+ {"Region": "South", "Revenue": " 4.25 "},
+ ],
+ form_data,
+ )
+ assert model.measures == [0.0, 4.25]
+
+
+@pytest.mark.parametrize(
+ ("presentation", "message"),
+ [
+ ({"ranges": "10,nope"}, r"ranges\[1\].*not numeric"),
+ ({"markers": "NaN"}, r"markers\[0\].*NaN or infinite"),
+ (
+ {"ranges": "10,20", "range_labels": "Only one"},
+ "one label per value",
+ ),
+ ],
+)
+def test_bullet_result_validation_rejects_malformed_presentation(
+ presentation: dict[str, object], message: str
+) -> None:
+ form_data = {
+ **map_bullet_config(
+ BulletChartConfig(metric={"name": "amount", "aggregate": "SUM"})
+ ),
+ **presentation,
+ }
+ with pytest.raises(BulletOutputError, match=message):
+ resolve_bullet_render_model([{"SUM(amount)": 1}], form_data)
+
+
+def test_bullet_compile_accepts_empty_ungrouped_result() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {"queries": [{"data": []}]}
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ assert result.success is True
+ assert result.row_count == 0
+
+
+def test_bullet_compile_inspects_top_level_and_query_error_envelopes() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(metric={"name": "Revenue", "aggregate": "SUM"})
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {
+ "status": "success",
+ "queries": [{"status": "failed", "message": "warehouse timeout"}],
+ }
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ assert result.success is False
+ assert "warehouse timeout" in (result.error or "")
+
+
+_MALFORMED_QUERY_ENVELOPES: list[object] = [
+ None,
+ [],
+ {},
+ {"queries": None},
+ {"queries": []},
+ {"queries": [None]},
+ {"queries": [{}]},
+ {"queries": [{"data": None}]},
+ {"queries": [{"data": []}, {"data": "not-an-array"}]},
+ {
+ "queries": [
+ {
+ "data": [{"Revenue": 12}],
+ "colnames": ["Revenue"],
+ "coltypes": [],
+ }
+ ]
+ },
+]
+
+
+def _compile_bullet_with_result(result: object) -> Any:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = result
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ return _compile_chart(form_data, 7)
+
+
+@pytest.mark.parametrize("envelope", _MALFORMED_QUERY_ENVELOPES)
+def test_bullet_compile_returns_stable_error_for_malformed_envelopes(
+ envelope: object,
+) -> None:
+ result = _compile_bullet_with_result(envelope)
+ assert result.success is False
+ assert result.error_code == "CHART_COMPILE_FAILED"
+ assert result.error_obj is not None
+ assert result.error_obj.error_type == "compile_error"
+
+
+@pytest.mark.parametrize(
+ ("data", "expected_code", "expected_type"),
+ [
+ ([1], "CHART_COMPILE_FAILED", "compile_error"),
+ (
+ [{"Revenue": 10**10000}],
+ "CHART_COMPILE_FAILED",
+ "compile_error",
+ ),
+ ],
+)
+def test_bullet_compile_returns_malformed_output_for_bad_rows(
+ data: list[object],
+ expected_code: str,
+ expected_type: str,
+) -> None:
+ result = _compile_bullet_with_result({"queries": [{"data": data}]})
+ assert result.success is False
+ assert result.error_code == expected_code
+ assert result.error_obj is not None
+ assert result.error_obj.error_type == expected_type
+
+
+def test_bullet_shared_query_builder_matches_frontend_build_query() -> None:
+ metric = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Region"}],
+ row_limit=25,
+ order_by=[{"column": "Revenue", "ascending": False}],
+ )
+ )
+ with patch(
+ "superset.mcp_service.chart.chart_helpers.resolve_datasource_engine",
+ return_value="base",
+ ):
+ queries = build_query_dicts_from_form_data(metric, 7, "table")
+ assert len(queries) == 1
+ assert queries[0]["columns"] == ["Region"]
+ assert queries[0]["metrics"] == [metric["metric"]]
+ assert queries[0]["orderby"] == metric["orderby"]
+ assert queries[0]["row_limit"] == 25
+
+
+def test_bullet_compile_path_uses_groupby_metric_orderby_and_usable_result() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Region"}],
+ order_by=[{"column": "Revenue", "ascending": False}],
+ )
+ )
+ context = MagicMock()
+ factory = MagicMock()
+ factory.create.return_value = context
+ command = MagicMock()
+ command.run.return_value = {
+ "queries": [{"data": [{"Region": "North", "Revenue": "12.5"}]}]
+ }
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ query = factory.create.call_args.kwargs["queries"][0]
+ assert query["columns"] == ["Region"]
+ assert query["metrics"] == [form_data["metric"]]
+ assert query["orderby"] == form_data["orderby"]
+ assert result.success is True
+ assert result.row_count == 1
+
+
+def test_bullet_compile_projects_dataframe_timestamp_before_validation() -> None:
+ from superset.dataframe import df_to_records
+
+ dublin = dateutil_tz.gettz("Europe/Dublin")
+ new_york = dateutil_tz.gettz("America/New_York")
+ assert dublin is not None
+ assert new_york is not None
+ source_values = [
+ pd.Timestamp("2024-01-02 03:04:05.123456789"),
+ datetime(2024, 10, 27, 1, 30, tzinfo=dublin, fold=1),
+ datetime(2024, 3, 10, 2, 30, tzinfo=new_york),
+ datetime(2040, 7, 1, 12, tzinfo=new_york),
+ ]
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Category"}],
+ )
+ )
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {"queries": [{"data": rows}]}
+ captured: list[list[dict[str, Any]]] = []
+
+ def validate(data: list[dict[str, Any]], config: dict[str, Any]) -> Any:
+ captured.append(data)
+ return resolve_bullet_render_model(data, config)
+
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ patch(
+ "superset.mcp_service.chart.preview_utils.resolve_bullet_render_model",
+ side_effect=validate,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+
+ assert result.success is True
+ assert [row["Category"] for row in captured[0]] == [
+ 1704164645123.456,
+ 1729989000000.0,
+ 1710052200000.0,
+ 2224774800000.0,
+ ]
+
+
+def test_bullet_compile_projects_real_dataframe_durations_to_chart_data_wire() -> None:
+ from superset.commands.chart.data.get_data_command import ChartDataCommand
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+ from superset.utils import json
+
+ source_values = [
+ timedelta(0),
+ timedelta(days=1, seconds=2, microseconds=3),
+ pd.Timedelta(-1, unit="ns"),
+ pd.Timedelta("1 days 00:00:02.000003004"),
+ np.timedelta64(123456789, "ns"),
+ np.timedelta64("NaT"),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Duration": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+ expected = [
+ None
+ if row["Duration"] is None
+ else json.loads(json.dumps(row["Duration"], default=json.json_int_dttm_ser))
+ for row in rows
+ ]
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Test crashes on np.timedelta64</b></div>
<div id="fix">
`json.json_int_dttm_ser` cannot serialize `np.timedelta64` (falls through `base_json_conv` and raises `TypeError`). Since `source_values` always includes `np.timedelta64(123456789, "ns")` and `np.timedelta64("NaT")`, and `df_to_records` leaves them as-is, this `expected` computation crashes the test. Use `_chart_data_duration_text` (imported here) to match the real wire projection, and remove the now-unused `json` import.
</div>
<details>
<summary>
<b>Code suggestion</b>
</summary>
<blockquote>Check the AI-generated fix before applying</blockquote>
<div id="code">
````suggestion
)
expected = []
for row in rows:
text, reason = _chart_data_duration_text(row["Duration"])
assert reason is None
expected.append(text)
````
</div>
</details>
</div>
<small><i>Code Review Run #d53793</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5660115102
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5497237287
<!-- Bito Reply -->
The issue described is a logic error in the Bullet chart configuration mapping where range rectangles lack a category `y` encoding. This causes thresholds to render as chart-wide bands instead of being scoped to individual bullet rows.
To resolve this, you need to ensure that the `range_rectangles` (or equivalent threshold configuration) in the `form_data` mapping includes the necessary `y` encoding or grouping information that links them to the specific bullet row. Since the file `superset/mcp_service/chart/tool/get_chart_preview.py` is not present in the provided PR context, I cannot implement the fix directly. Please verify the mapping logic in that file to ensure the `y` encoding is correctly passed to the frontend configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5801837507
@aminghadersohi thanks for flagging this instead of slipping it in. Since it touches response shape on `get_chart_data`, `get_dashboard_data`, `query_dataset`, `get_table` and `execute_sql`, not just Bullet, I'd rather see it split into its own PR with the UPDATING.md note. Easier to review and easier to revert if the string-shape change needs another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5690124148
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4171778769
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1750,523 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
+ (form_data.get("all_columns") or [])
+ if raw_mode
+ else (form_data.get("groupby") or [])
+ )
+ table_metrics = [] if raw_mode else list(form_data.get("metrics") or [])
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ query["columns"] = [
+ _temporal_column(column, form_data) for column in table_columns
+ ]
+ offsets = _table_time_offsets(form_data, query)
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if offsets:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ configured_limit = form_data.get("row_limit")
+ if form_data.get("server_pagination"):
+ page_size = form_data.get("server_page_length") or 0
+ if page_size:
Review Comment:
This replaces the normalized caller limit with the UI page size: a saved row_limit of "250" raises TypeError here, while a CSV/Excel request for 1,000 rows is reduced to a 25-row page. Could this use the normalized limit and apply page sizing only to non-download queries?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the selected roles. Its zero-valued
+ # ungrouped row is a render-only frontend fallback, not source query
+ # data, so an empty query exposes no rows to get-data and exports.
+ return (model.rows if data else []), None
Review Comment:
Returning the render-model rows changes an exact BIGINT metric such as "9007199254740993" into 9007199254740992.0 before JSON and CSV/Excel export, silently changing the source value. Could get-data/export retain the validated source metric and use the float measure only for rendering?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1781,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+ """
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
Review Comment:
A Bullet saved in Explore can have an OrderDate TEMPORAL_RANGE filter without an MCP provenance marker; a time_range-only update appends a second OrderDate filter and marks that subject, so the next update raises "found 2" before even handling a filter clear. Could the range update replace the existing native binding instead of persisting duplicate marker-owned filters?
##########
superset/mcp_service/chart/tool/update_chart.py:
##########
@@ -363,45 +398,88 @@ def _inherited_state_invalid_keys(
return invalid_keys
+def _prune_inherited_query_state(
+ existing_form_data: dict[str, Any],
+ new_form_data: dict[str, Any],
+ config: ChartConfig,
+ dataset_id: int,
+) -> dict[str, Any]:
+ """Discard incompatible inherited roles and their temporal provenance."""
+ invalid_keys = _inherited_state_invalid_keys(
+ existing_form_data, new_form_data, config, dataset_id
+ )
+ if "adhoc_filters" in invalid_keys:
+ invalid_keys.add(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ return {
+ key: value
+ for key, value in existing_form_data.items()
+ if key not in invalid_keys
+ }
+
+
def _build_replacement_form_data(
existing_form_data: dict[str, Any],
parsed_config: ChartConfig,
effective_dataset_id: int | None,
replacement_dataset_id: int | None = None,
) -> dict[str, Any]:
"""Map and merge a replacement config for preview and save paths."""
+ existing_plugin = plugin_for_viz_type(existing_form_data.get("viz_type"))
+ include_disabled = bool(
+ existing_plugin and existing_plugin.chart_type == parsed_config.chart_type
+ )
new_form_data = map_config_to_form_data(
- parsed_config, dataset_id=effective_dataset_id, include_disabled=True
+ parsed_config,
+ dataset_id=effective_dataset_id,
+ include_disabled=include_disabled,
)
new_form_data.pop("_mcp_warnings", None)
dataset_rebind = replacement_dataset_id is not None
- config_plugin = get_registry().get(parsed_config.chart_type, include_disabled=True)
+ config_plugin = get_registry().get(
+ parsed_config.chart_type, include_disabled=include_disabled
+ )
if replacement_dataset_id is not None and not (
config_plugin is not None and config_plugin.strict_dataset_rebind
):
# Drop only the inherited state the replacement dataset cannot
# resolve, then merge as a same-dataset update. Plugins with a strict
# rebind contract handle the rebind in merge_update_form_data.
- invalid_keys = _inherited_state_invalid_keys(
- existing_form_data,
- new_form_data,
- parsed_config,
- replacement_dataset_id,
+ existing_form_data = _prune_inherited_query_state(
+ existing_form_data, new_form_data, parsed_config, replacement_dataset_id
)
- existing_form_data = {
- key: value
- for key, value in existing_form_data.items()
- if key not in invalid_keys
- }
dataset_rebind = False
merge_table_column_config(existing_form_data, new_form_data)
merge_interactive_pivot_ui_config(existing_form_data, new_form_data)
- merged = _merge_replacement_config(
- existing_form_data,
- new_form_data,
- parsed_config,
- dataset_rebind=dataset_rebind,
- )
+ merge_plugin = plugin_for_viz_type(new_form_data.get("viz_type"))
+ if merge_plugin is not None and merge_plugin.owns_update_merge:
+ # The plugin owns its complete merge contract, including temporal
+ # predicates and dataset rebinds. Generic update merges would restore
+ # inherited state that the contract deliberately removed.
+ merged = _merge_replacement_config(
+ existing_form_data,
+ new_form_data,
+ parsed_config,
+ dataset_rebind=dataset_rebind,
+ )
+ else:
+ merged = _merge_replacement_config(
+ existing_form_data,
+ new_form_data,
+ parsed_config,
+ dataset_rebind=dataset_rebind,
+ )
+ merge_update_form_data(existing_form_data, merged, parsed_config)
Review Comment:
Changing a US-filtered Table to Pie while omitting filters now restores the old Table predicate here, after merge_chart_form_data deliberately discarded state from the previous visualization, so the replacement Pie still queries only US rows. Could this follow-up filter merge respect the same visualization boundary on both update paths?
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,427 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ matches = [
+ item
+ for item in saved_metrics
+ if item == name or item.casefold() == name.casefold()
+ ]
+ if len(set(matches)) != 1:
Review Comment:
With saved metrics named Revenue and revenue, an exact Revenue reference collects both case-folded matches and is rejected before compilation, although native metric lookup selects the exact name unambiguously. Could exact matches take precedence here, reserving the ambiguity check for non-exact lookups?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1781,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
Review Comment:
Clearing a sorted Bullet with dimensions: [] leaves the saved orderby targeting Region in this merge, so final validation rejects the update even though the requested ungrouped chart is valid. Could ordering entries tied to removed dimensions be cleared along with those roles, while retaining unaffected metric sorts?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1750,523 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
+ (form_data.get("all_columns") or [])
+ if raw_mode
+ else (form_data.get("groupby") or [])
+ )
+ table_metrics = [] if raw_mode else list(form_data.get("metrics") or [])
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ query["columns"] = [
+ _temporal_column(column, form_data) for column in table_columns
+ ]
+ offsets = _table_time_offsets(form_data, query)
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if offsets:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ configured_limit = form_data.get("row_limit")
+ if form_data.get("server_pagination"):
+ page_size = form_data.get("server_page_length") or 0
+ if page_size:
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if table_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ table_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
+ columns = _resolve_big_number_query_columns(form_data) if trendline else []
+ query = build_single_query_dict(
+ form_data,
+ columns,
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if trendline:
+ # Big Number has no series dimension. Its frontend pivot receives
+ # the common base QueryObject (whose columns are empty), not the
+ # final QueryObject after the explicit x-axis is added. Preserve
+ # that distinction instead of falling back to the final columns.
+ query["series_columns"] = []
+ if not form_data.get("x_axis"):
+ query["is_timeseries"] = True
+ query["post_processing"] = _timeseries_post_processing(form_data, query)
+ if form_data.get("aggregation") == "raw":
+ return [
+ query,
+ {
+ **query,
+ "columns": [],
+ "is_timeseries": False,
+ "post_processing": [],
+ },
+ ]
+ return [query]
+
+
+def build_waterfall_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Waterfall buildQuery with raw-axis ordering."""
+ metrics, groupby = resolve_metrics_and_groupby(form_data)
+ # normalizeTimeColumn runs after Waterfall's buildQuery callback. It
+ # wraps only the final x-axis column; orderby deliberately retains the
+ # raw control value produced inside the callback.
+ raw_axis = form_data.get("x_axis") or form_data.get("granularity_sqla")
+ query_axis = (
+ _normalized_x_axis_query_field(form_data)
+ if form_data.get("x_axis")
+ else raw_axis
+ )
+ waterfall_columns = ([query_axis] if query_axis else []) + groupby
+ raw_ordering_columns = ([raw_axis] if raw_axis else []) + groupby
+ query = build_single_query_dict(
+ form_data,
+ waterfall_columns,
+ metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=None,
+ )
+ query["orderby"] = [[column, True] for column in raw_ordering_columns]
+ if form_data.get("x_axis"):
+ query.pop("is_timeseries", None)
+ return [query]
+
+
+def build_mixed_timeseries_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Mixed Timeseries buildQuery for both independent layers."""
+ from superset.utils.core import split_adhoc_filters_into_base_filters
+
+ queries: list[dict[str, Any]] = []
+ x_axis = _normalized_x_axis_query_field(form_data)
+ for secondary in (False, True):
+ layer = _mixed_layer_form_data(form_data, secondary=secondary)
+ if secondary and form_data.get("adhoc_filters_b") is not None:
+ for key in ("filters", "where", "having"):
+ layer.pop(key, None)
+ layer["adhoc_filters"] = form_data.get("adhoc_filters_b") or []
Review Comment:
When extra_form_data.adhoc_filters supplies a Region = North predicate and adhoc_filters_b is empty, preparation appends that predicate to the common filters, but this reset removes it from query B. The response can therefore combine North-only sales with costs for every region; could request-level adhoc filters be applied to both layers after their local filters are selected?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the selected roles. Its zero-valued
+ # ungrouped row is a render-only frontend fallback, not source query
+ # data, so an empty query exposes no rows to get-data and exports.
+ return (model.rows if data else []), None
+
+ def ascii_preview(
+ self, data: list[Any], form_data: dict[str, Any], width: int
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def vega_lite_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> VegaLitePreview | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_bullet_vega_lite_preview,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_bullet_vega_lite_preview(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def resolve_update_config(
+ self,
+ config: Any,
+ existing_form_data: dict[str, Any],
+ *,
+ dataset_rebind: bool,
+ ) -> Any:
+ """Resolve sort targets against the saved hierarchy before mapping."""
+ if not isinstance(config, BulletChartConfig) or config.dimensions is not None:
+ return config
+ if not config.order_by:
+ return config
+ dimensions = (
+ existing_form_data.get("groupby") or []
+ if not dataset_rebind and existing_form_data.get("viz_type") == "bullet"
+ else []
+ )
+ return BulletChartConfig.model_validate(
Review Comment:
During a saved-chart dataset change, supplying metric-based order_by while omitting dimensions reconstructs the config with dimensions: [], so a compatible saved Region hierarchy is silently cleared; the cached-preview path preserves it. Could saved updates resolve the omitted hierarchy against the replacement dataset before treating it as an explicit clear?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1857 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
return None
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
Review Comment:
A successful query containing a BYTEA/BLOB value reaches this gate as bytes or memoryview, which has no supported scalar branch here and becomes MalformedQueryResult before the existing UTF-8/base64 serializer can run. Could the bounded validator preserve the documented binary-column contract for query_dataset, get_table and chart data?
##########
tests/unit_tests/mcp_service/chart/tool/test_get_chart_data.py:
##########
@@ -2530,106 +2990,2240 @@ def test_multi_query_row_count_reflects_limit() -> None:
]
-@pytest.mark.asyncio
-async def test_unsaved_mixed_timeseries_returns_nonempty_secondary_query(
+def _multi_query_chart_data(
+ query_results: list[dict[str, Any]],
+) -> ChartData:
+ nested_results = _build_query_results(query_results, None)
+ assert nested_results is not None
+ first_data = query_results[0]["data"]
+ return ChartData(
+ chart_id=1,
+ chart_name="Multi-query",
+ chart_type="mixed_timeseries",
+ columns=[],
+ data=first_data,
+ query_results=nested_results,
+ row_count=len(first_data),
+ total_rows=len(first_data),
+ summary="Multi-query result",
+ insights=[],
+ data_quality={},
+ recommended_visualizations=[],
+ data_freshness=None,
+ performance=PerformanceMetadata(query_duration_ms=1, cache_status="fresh"),
+ )
+
+
+def test_complete_response_budget_charges_multi_query_first_leg_alias(
monkeypatch: pytest.MonkeyPatch,
) -> None:
- """The tool response must not collapse a multi-query command result."""
- from unittest.mock import AsyncMock
-
- get_data_command_module = importlib.import_module(
- "superset.commands.chart.data.get_data_command"
- )
- chart_data_module = importlib.import_module(
- "superset.mcp_service.chart.tool.get_chart_data"
+ row = {f"c{index}": 0 for index in range(20)}
+ rows = [row] * 100
+ query_results = [
+ {"colnames": list(row), "data": rows, "rowcount": len(rows)},
+ {"colnames": list(row), "data": rows, "rowcount": len(rows)},
+ ]
+ source = {"queries": query_results}
+ source_bytes = len(json.dumps(source, separators=(",", ":")).encode())
+ monkeypatch.setattr(
+ importlib.import_module("superset.mcp_service.chart.query_result"),
+ "MAX_QUERY_RESULT_VALUE_BYTES",
+ source_bytes,
)
- class MultiQueryChartDataCommand:
+ data, source_failure = query_result_data(source)
+ assert source_failure is None
+ assert data == [rows, rows]
+
+ response = _multi_query_chart_data(query_results)
+ assert len(response.model_dump_json().encode()) > source_bytes
+ response_failure = response_json_failure(response)
+
+ assert response_failure is not None
+ assert "response exceeds the total JSON-encoded byte limit" in (
+ response_failure.error
+ )
+
+
+def test_complete_response_preflight_matches_real_pydantic_serialization() -> None:
+ rows = [{"label": 'café "\n', "value": index} for index in range(100)]
+ query_results = [
+ {"colnames": ["label", "value"], "data": rows, "rowcount": len(rows)},
+ {"colnames": ["label", "value"], "data": rows, "rowcount": len(rows)},
+ ]
+ response = _multi_query_chart_data(query_results)
+
+ assert response_json_failure(response) is None
+ assert len(response.model_dump_json().encode()) <= MAX_QUERY_RESULT_VALUE_BYTES
+
+
+@pytest.mark.asyncio
+async def test_unsaved_mixed_timeseries_returns_nonempty_secondary_query(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """The tool response must not collapse a multi-query command result."""
+ from unittest.mock import AsyncMock
+
+ get_data_command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart_data_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_data"
+ )
+
+ class MultiQueryChartDataCommand:
+ def __init__(self, query_context: object) -> None:
+ self.query_context = query_context
+
+ def validate(self) -> None:
+ pass
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "colnames": ["primary"],
+ "data": [],
+ "rowcount": 0,
+ },
+ {
+ "colnames": ["secondary"],
+ "data": [{"secondary": 2}],
+ "rowcount": 1,
+ },
+ ]
+ }
+
+ monkeypatch.setattr(
+ chart_data_module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(
+ get_data_command_module, "ChartDataCommand", MultiQueryChartDataCommand
+ )
+ request = GetChartDataRequest(form_data_key="mixed-chart")
+ response = await _query_from_form_data(
+ {
+ "datasource_id": 1,
+ "datasource_type": "table",
+ "viz_type": "mixed_timeseries",
+ "row_limit": 10,
+ },
+ request,
+ AsyncMock(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.data == []
+ assert response.query_results is not None
+ assert [result.data for result in response.query_results] == [
+ [],
+ [{"secondary": 2}],
+ ]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ "payload",
+ [
+ {"queries": None},
+ {"queries": []},
+ {"queries": [{}]},
+ {"queries": [{"data": None}]},
+ {"queries": [{"data": []}, {}]},
+ {"queries": [{"data": [{"value": 1}], "colnames": ["value"], "coltypes": []}]},
+ ],
+)
+async def test_unsaved_chart_data_rejects_malformed_query_envelopes(
+ monkeypatch: pytest.MonkeyPatch, payload: dict[str, Any]
+) -> None:
+ from unittest.mock import AsyncMock
+
+ chart_data_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_data"
+ )
+ get_data_command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class MalformedChartDataCommand:
def __init__(self, query_context: object) -> None:
self.query_context = query_context
- def validate(self) -> None:
- pass
+ def validate(self) -> None:
+ pass
+
+ def run(self) -> dict[str, Any]:
+ return payload
+
+ monkeypatch.setattr(
+ chart_data_module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(
+ chart_data_module,
+ "event_logger",
+ SimpleNamespace(log_context=lambda **_kwargs: nullcontext()),
+ )
+ monkeypatch.setattr(
+ get_data_command_module, "ChartDataCommand", MalformedChartDataCommand
+ )
+
+ response = await _query_from_form_data(
+ {"datasource_id": 1, "datasource_type": "table"},
+ GetChartDataRequest(form_data_key="malformed"),
+ AsyncMock(),
+ )
+
+ assert isinstance(response, ChartError)
+ assert response.error_type == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+async def test_unsaved_chart_data_accepts_valid_empty_dataset(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ from unittest.mock import AsyncMock
+
+ chart_data_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_data"
+ )
+ get_data_command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ command = MagicMock()
+ command.run.return_value = {"queries": [{"data": [], "colnames": []}]}
+ monkeypatch.setattr(
+ chart_data_module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(
+ chart_data_module,
+ "event_logger",
+ SimpleNamespace(log_context=lambda **_kwargs: nullcontext()),
+ )
+ monkeypatch.setattr(
+ get_data_command_module, "ChartDataCommand", lambda _context: command
+ )
+
+ response = await _query_from_form_data(
+ {"datasource_id": 1, "datasource_type": "table"},
+ GetChartDataRequest(form_data_key="empty"),
+ AsyncMock(),
+ )
+
+ assert isinstance(response, ChartError)
+ assert response.error_type == "NoData"
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ "row",
+ [
+ _HostileResultRow(value=1),
+ {"value": _HostileResultScalar("unsafe")},
+ object(),
+ ],
+)
+async def test_unsaved_get_data_rejects_hostile_rows_and_scalars(
+ monkeypatch: pytest.MonkeyPatch,
+ row: Any,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {"queries": [{"data": [row], "colnames": ["value"]}]}
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_a, **_k: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="hostile"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartError)
+ assert response.error_type == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+async def test_chart_metadata_distinguishes_booleans_from_equal_integers(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ values = (True, 1, False, 0)
+ return {
+ "queries": [
+ {
+ "data": [{"value": value} for value in values],
+ "colnames": ["value"],
+ "coltypes": [GenericDataType.BOOLEAN],
+ "rowcount": len(values),
+ }
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="boolean-identity"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.columns[0].data_type == "boolean"
+ assert response.columns[0].unique_count == 4
+
+
+def test_numeric_metadata_identity_matches_bounded_value_semantics() -> None:
+ equivalent_groups = [
+ [0, -0.0, 0.0, Decimal("0"), Decimal("-0.000")],
+ [1, 1.0, Decimal("1"), Decimal("1.0"), Decimal("1.00")],
+ [0.5, Decimal("0.5"), Decimal("0.50")],
+ [float("inf"), Decimal("Infinity")],
+ [float("-inf"), Decimal("-Infinity")],
+ ]
+
+ for group in equivalent_groups:
+ identities = [_safe_value_identity(value) for value in group]
+ assert len(set(identities)) == 1
+
+ specials = [
+ Decimal("NaN"),
+ Decimal("-NaN42"),
+ Decimal("sNaN"),
+ Decimal("-sNaN7"),
+ float("nan"),
+ ]
+ identities = [_safe_value_identity(value) for value in specials]
+ assert len(set(identities)) == len(specials)
+ assert identities == [_safe_value_identity(value) for value in specials]
+
+
+def test_aware_datetime_and_time_identity_preserves_instant_semantics() -> None:
+ plus_one = timezone(timedelta(hours=1))
+ same_datetimes = [
+ datetime(2024, 1, 1, 12, tzinfo=timezone.utc),
+ datetime(2024, 1, 1, 13, tzinfo=plus_one),
+ ]
+ distinct_datetime = datetime(2024, 1, 1, 12, tzinfo=plus_one)
+ same_times = [
+ datetime_time(12, tzinfo=timezone.utc),
+ datetime_time(13, tzinfo=plus_one),
+ ]
+ distinct_time = datetime_time(12, tzinfo=plus_one)
+
+ assert len({_safe_value_identity(value) for value in same_datetimes}) == 1
+ assert _safe_value_identity(distinct_datetime) != _safe_value_identity(
+ same_datetimes[0]
+ )
+ assert len({_safe_value_identity(value) for value in same_times}) == 1
+ assert _safe_value_identity(distinct_time) != _safe_value_identity(same_times[0])
+
+
+def test_zoneinfo_time_without_offset_uses_python_naive_identity() -> None:
+ zone = ZoneInfo("America/New_York")
+ naive = datetime_time(12, 34, 56, 789)
+ zoned = datetime_time(12, 34, 56, 789, tzinfo=zone)
+ folded = datetime_time(12, 34, 56, 789, tzinfo=zone, fold=1)
+
+ assert zoned.utcoffset() is None
+ assert zoned == naive == folded
+ assert _safe_value_identity(zoned) == _safe_value_identity(naive)
+ assert _safe_value_identity(folded) == _safe_value_identity(naive)
+
+
+def test_zoneinfo_datetime_fold_matches_python_same_zone_semantics() -> None:
+ zone = ZoneInfo("America/New_York")
+ first = datetime(2024, 11, 3, 1, 30, tzinfo=zone, fold=0)
+ second = datetime(2024, 11, 3, 1, 30, tzinfo=zone, fold=1)
+
+ assert first.utcoffset() != second.utcoffset()
+ assert first == second
+ assert hash(first) == hash(second)
+ assert _safe_value_identity(first) == _safe_value_identity(second)
+
+
+def test_zoneinfo_interzone_ambiguity_matches_python_equality_and_hash() -> None:
+ new_york = ZoneInfo("America/New_York")
+ toronto = ZoneInfo("America/Toronto")
+ ambiguous = datetime(2024, 11, 3, 1, 30, tzinfo=new_york, fold=0)
+ same_utc_instant = datetime(2024, 11, 3, 5, 30, tzinfo=timezone.utc)
+ other_zone = datetime(2024, 11, 3, 1, 30, tzinfo=toronto, fold=0)
+ unambiguous = datetime(2024, 11, 3, 3, 30, tzinfo=new_york)
+ unambiguous_utc = datetime(2024, 11, 3, 8, 30, tzinfo=timezone.utc)
+
+ # PEP 495 makes an offset-dependent ambiguous value unequal across zones,
+ # even when one selected fold denotes the same UTC instant.
+ assert ambiguous != same_utc_instant
+ assert ambiguous != other_zone
+ assert _safe_value_identity(ambiguous) != _safe_value_identity(same_utc_instant)
+ assert _safe_value_identity(ambiguous) != _safe_value_identity(other_zone)
+
+ assert unambiguous == unambiguous_utc
+ assert hash(unambiguous) == hash(unambiguous_utc)
+ assert _safe_value_identity(unambiguous) == _safe_value_identity(unambiguous_utc)
+
+
+def test_time_fold_and_fixed_offset_identity_matches_python() -> None:
+ zone = ZoneInfo("America/New_York")
+ zoned_fold_zero = datetime_time(1, 30, tzinfo=zone, fold=0)
+ zoned_fold_one = datetime_time(1, 30, tzinfo=zone, fold=1)
+ naive = datetime_time(1, 30)
+ utc = datetime_time(12, tzinfo=timezone.utc)
+ plus_one = datetime_time(13, tzinfo=timezone(timedelta(hours=1)))
+
+ assert zoned_fold_zero == zoned_fold_one == naive
+ assert hash(zoned_fold_zero) == hash(zoned_fold_one) == hash(naive)
+ assert (
+ _safe_value_identity(zoned_fold_zero)
+ == _safe_value_identity(zoned_fold_one)
+ == _safe_value_identity(naive)
+ )
+ assert utc == plus_one
+ assert hash(utc) == hash(plus_one)
+ assert _safe_value_identity(utc) == _safe_value_identity(plus_one)
+
+
+def test_opaque_timezone_identity_does_not_execute_custom_offset() -> None:
+ class _HostileTimezone(tzinfo):
+ def utcoffset(self, _value: datetime | None) -> timedelta | None:
+ raise AssertionError("custom timezone hook must not execute")
+
+ def dst(self, _value: datetime | None) -> timedelta | None:
+ raise AssertionError("custom timezone hook must not execute")
+
+ def tzname(self, _value: datetime | None) -> str | None:
+ raise AssertionError("custom timezone hook must not execute")
+
+ hostile = _HostileTimezone()
+ value = datetime(2024, 1, 1, 12, tzinfo=hostile)
+ time_value = datetime_time(12, tzinfo=hostile)
+
+ assert _safe_value_identity(value) == _safe_value_identity(value)
+ assert _safe_value_identity(time_value) == _safe_value_identity(time_value)
+
+
+def test_oversized_numeric_identity_declines_unbounded_conversion() -> None:
+ huge_int = 1 << 100_000
+ scale_variant_a = Decimal("1e1000000")
+ scale_variant_b = Decimal("10e999999")
+ huge_decimal = Decimal("1" * 100_000)
+
+ int_identity = _safe_value_identity(huge_int)
+ decimal_identity = _safe_value_identity(huge_decimal)
+
+ assert int_identity[:2] == ("oversized_numeric", "integer")
+ assert decimal_identity[:2] == ("oversized_numeric", "decimal")
+ assert _safe_value_identity(scale_variant_a) == _safe_value_identity(
+ scale_variant_b
+ )
+ assert len(repr(int_identity)) < 200
+ assert len(repr(decimal_identity)) < 200
+
+
+def test_mixed_numeric_unique_count_uses_value_semantics() -> None:
+ values = [
+ 0,
+ -0.0,
+ Decimal("-0.00"),
+ 1,
+ 1.0,
+ Decimal("1.000"),
+ Decimal("sNaN"),
+ Decimal("sNaN"),
+ Decimal("NaN"),
+ float("inf"),
+ Decimal("Infinity"),
+ ]
+
+ assert len({_safe_value_identity(value) for value in values}) == 5
+
+
+def test_chart_column_metadata_bounds_wide_sparse_statistics_work() -> None:
+ from superset.mcp_service.chart.tool.get_chart_data import _build_data_columns
+ from superset.mcp_service.utils.response_utils import (
+ data_column_stats_row_limit,
+ )
+
+ rows: list[dict[str, Any]] = [{} for _ in range(50_000)]
+ raw_columns = [f"column_{index}" for index in range(4096)]
+
+ columns = _build_data_columns(rows, raw_columns, [])
+
+ sampled_rows = data_column_stats_row_limit(len(rows), len(raw_columns))
+ assert len(columns) == len(raw_columns)
+ assert columns[0].statistics == {"sampled_rows": sampled_rows}
+ assert columns[-1].statistics == {"sampled_rows": sampled_rows}
+
+
+@pytest.mark.asyncio
+async def test_unsaved_generic_get_data_returns_finite_decimal_metadata(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ values = [
+ Decimal("1.25"),
+ Decimal("1.250"),
+ 1.25,
+ 0,
+ -0.0,
+ Decimal("-0.00"),
+ ]
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": value} for value in values],
+ "colnames": ["value"],
+ "rowcount": len(values),
+ }
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="decimal"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.columns[0].data_type == "numeric"
+ assert response.columns[0].unique_count == 2
+ assert response.columns[0].sample_values == values[:3]
+
+
+@pytest.mark.asyncio
+async def test_unsaved_get_data_canonicalizes_decimal_nonfinite_at_producer(
+ mcp_server: Any,
+ mock_auth: Any,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """The public MCP ChartData wire never receives Decimal non-finite tokens."""
+ from fastmcp import Client
+
+ from superset.dataframe import df_to_records
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ form_data = {
+ "datasource_id": 7,
+ "datasource_type": "table",
+ "datasource": "7__table",
+ "viz_type": "table",
+ "all_columns": ["value"],
+ }
+ finite = Decimal("0.10000000000000000001")
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "value": pd.Series(
+ [
+ Decimal("NaN"),
+ Decimal("sNaN"),
+ Decimal("Infinity"),
+ Decimal("-Infinity"),
+ finite,
+ ],
+ dtype=object,
+ )
+ }
+ ),
+ convert_big_integers=False,
+ )
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["value"],
+ "coltypes": [GenericDataType.NUMERIC],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+ monkeypatch.setattr(
+ module, "get_cached_form_data", lambda _key: json.dumps(form_data)
+ )
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(module, "set_query_context_form_data", lambda *_args: None)
+ monkeypatch.setattr(
+ module,
+ "event_logger",
+ SimpleNamespace(log_context=lambda **_kwargs: nullcontext()),
+ )
+ monkeypatch.setattr(module.guest_scope, "is_guest_read", lambda: False)
+
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"form_data_key": "decimal-nonfinite"}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert [row["value"] for row in payload["data"]] == [
+ None,
+ None,
+ None,
+ None,
+ "0.10000000000000000001",
+ ]
+
+
+@pytest.mark.asyncio
+async def test_unsaved_get_data_reports_sampled_all_null_completeness(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ row_count = 10_000
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": float("nan")} for _ in range(row_count)],
+ "colnames": ["value"],
+ "coltypes": [GenericDataType.NUMERIC],
+ "rowcount": row_count,
+ }
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="nulls"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.columns[0].null_count == 5000
+ assert response.columns[0].statistics == {"sampled_rows": 5000}
+ assert response.data_quality == {
+ "completeness": 0.0,
+ "completeness_is_approximate": True,
+ "sampled_rows": 5000,
+ }
+ assert all(row["value"] is None for row in response.data)
+
+
+@pytest.mark.asyncio
+async def test_unsaved_get_data_round_trips_temporal_and_boolean_coltypes(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [
+ {
+ "event_time": datetime(
+ 2024,
+ 1,
+ 1,
+ 2,
+ 3,
+ 4,
+ tzinfo=dateutil_tz.tzoffset(
+ "east", 5 * 3600 + 30 * 60
+ ),
+ ),
+ "clock_time": datetime_time(
+ 2, 3, 4, tzinfo=pytz.FixedOffset(-450)
+ ),
+ "enabled": True,
+ "missing": np.float64("nan"),
+ }
+ ],
+ "colnames": [
+ "event_time",
+ "clock_time",
+ "enabled",
+ "missing",
+ ],
+ "coltypes": [
+ GenericDataType.TEMPORAL,
+ GenericDataType.TEMPORAL,
+ GenericDataType.BOOLEAN,
+ GenericDataType.NUMERIC,
+ ],
+ "rowcount": 1,
+ }
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="typed"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert [column.data_type for column in response.columns] == [
+ "temporal",
+ "temporal",
+ "boolean",
+ "numeric",
+ ]
+ assert response.data == [
+ {
+ "event_time": "2024-01-01T02:03:04+05:30",
+ "clock_time": "02:03:04-07:30",
+ "enabled": True,
+ "missing": None,
+ }
+ ]
+
+
+@pytest.mark.asyncio
+async def test_unsaved_multi_query_consumes_canonical_cached_dttm(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ cached_dttm = (datetime.now(timezone.utc) - timedelta(minutes=90)).isoformat()
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": 1}],
+ "colnames": ["value"],
+ "rowcount": 1,
+ "is_cached": True,
+ "cached_dttm": cached_dttm,
+ },
+ {
+ "data": [{"other": 2}],
+ "colnames": ["other"],
+ "rowcount": 1,
+ "is_cached": True,
+ "cached_dttm": cached_dttm,
+ },
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="cached"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.cache_status is not None
+ assert response.cache_status.cache_hit is True
+ assert response.cache_status.cache_age_seconds is not None
+ assert response.cache_status.cache_age_seconds >= 5_399
+ assert response.query_results is not None
+ assert len(response.query_results) == 2
+
+
+@pytest.mark.asyncio
+async def test_saved_get_data_accepts_real_postprocessing_null_and_large_full_sql(
+ mcp_server: Any, mock_auth: Any, app_context: None
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ from tests.unit_tests.mcp_service.chart.query_result_test_utils import (
+ real_compare_command_result,
+ )
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=13,
+ slice_name="Comparison",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+ sql = 'SELECT "chart café"\\n' + "x" * (70 * 1024)
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return real_compare_command_result(sql)
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 13}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["data"][0]["finite"] == 2.5
+ assert any(value is None for value in payload["data"][0].values())
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("engine", ["openpyxl", "xlsxwriter"])
+async def test_real_get_chart_data_excel_stringifies_uuid_with_both_engines(
+ engine: str, mcp_server: Any, mock_auth: Any
+) -> None:
+ import base64
+ import io
+ from contextlib import ExitStack
+ from unittest.mock import patch
+
+ from fastmcp import Client
+ from openpyxl import load_workbook
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ identifier = UUID("12345678-1234-5678-1234-567812345678")
+ chart = SimpleNamespace(
+ id=15,
+ slice_name="UUID export",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"id": identifier}],
+ "colnames": ["id"],
+ "rowcount": 1,
+ }
+ ]
+ }
+
+ with ExitStack() as stack:
+ stack.enter_context(
+ patch.object(module, "find_chart_by_identifier", return_value=chart)
+ )
+ stack.enter_context(
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ )
+ )
+ stack.enter_context(
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ )
+ )
+ stack.enter_context(patch.object(command_module, "ChartDataCommand", _Command))
+ if engine == "xlsxwriter":
+ stack.enter_context(
+ patch.object(
+ module, "_create_excel_with_openpyxl", side_effect=ImportError
+ )
+ )
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {"request": {"identifier": 15, "format": "excel"}},
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["format"] == "excel"
+ workbook = load_workbook(io.BytesIO(base64.b64decode(payload["excel_data"])))
+ assert workbook.active["A2"].value == str(identifier)
+
+
+@pytest.mark.asyncio
+async def test_saved_get_data_uses_production_cache_timestamp_for_insights(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=14,
+ slice_name="Cached producer envelope",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+ cached_dttm = (datetime.now(timezone.utc) - timedelta(hours=2)).isoformat()
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": 1}],
+ "colnames": ["value"],
+ "rowcount": 1,
+ "is_cached": True,
+ "cached_dttm": cached_dttm,
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 14}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["cache_status"]["cache_hit"] is True
+ assert payload["cache_status"]["cache_age_seconds"] >= 7_199
+ assert any("2h old" in insight for insight in payload["insights"])
+ assert payload["performance"]["cache_status"] == "cache_hit"
+
+
+@pytest.mark.asyncio
+async def test_unsaved_bullet_get_data_uses_strict_render_model(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {"data": [{"Revenue": "not numeric"}], "colnames": ["Revenue"]}
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_a, **_k: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {
+ "datasource": "1__table",
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ },
+ GetChartDataRequest(form_data_key="bullet"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartError)
+ assert response.error_type == "MalformedBulletOutput"
+
+
+def _make_chart_data(**overrides: Any) -> ChartData:
+ """Build a minimal valid ChartData for testing."""
+ from superset.mcp_service.common.cache_schemas import CacheStatus
+
+ defaults: dict[str, Any] = {
+ "chart_id": 1,
+ "chart_name": "Test Chart",
+ "chart_type": "table",
+ "columns": [],
+ "data": [{"metric": 3.5}],
+ "row_count": 1,
+ "total_rows": 1,
+ "data_freshness": None,
+ "summary": "summary",
+ "insights": [],
+ "data_quality": {},
+ "recommended_visualizations": [],
+ "performance": PerformanceMetadata(
+ query_duration_ms=42,
+ cache_status="fresh_query",
+ ),
+ "cache_status": CacheStatus(cache_hit=False),
+ }
+ defaults.update(overrides)
+ return ChartData(**defaults)
+
+
+class TestChartDataTotalRowsCoercion:
+ """Regression tests: float total_rows causes PydanticSerializationError."""
+
+ def test_float_total_rows_is_coerced_to_int(self) -> None:
+ chart_data = _make_chart_data(total_rows=5.0)
+ assert chart_data.total_rows == 5
+ assert isinstance(chart_data.total_rows, int)
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ "row",
+ [
+ _HostileResultRow(value=1),
+ {"value": _HostileResultScalar("unsafe")},
+ object(),
+ ],
+)
+async def test_saved_get_data_rejects_hostile_rows_and_scalars(
+ mcp_server: Any,
+ mock_auth: Any,
+ row: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=9,
+ slice_name="Hostile rows",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {"queries": [{"data": [row], "colnames": ["value"]}]}
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 9}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["error_type"] == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+async def test_saved_generic_get_data_rejects_misaligned_coltypes(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=12,
+ slice_name="Malformed metadata",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": 1}],
+ "colnames": ["value"],
+ "coltypes": [],
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 12}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["error_type"] == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+async def test_saved_generic_get_data_returns_finite_decimal_metadata(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=11,
+ slice_name="Decimal metadata",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+ values = [
+ Decimal("1.25"),
+ Decimal("1.250"),
+ 1.25,
+ 0,
+ -0.0,
+ Decimal("-0.00"),
+ ]
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": value} for value in values],
+ "colnames": ["value"],
+ "rowcount": len(values),
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 11}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["columns"][0]["data_type"] == "numeric"
+ assert payload["columns"][0]["unique_count"] == 2
+ assert payload["columns"][0]["sample_values"] == ["1.25", "1.250", 1.25]
+
+
+@pytest.mark.asyncio
+async def test_saved_get_data_round_trips_temporal_and_boolean_coltypes(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=13,
+ slice_name="Typed metadata",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [
+ {
+ "event_time": pd.Timestamp(
+ datetime(
+ 2024,
+ 1,
+ 1,
+ 2,
+ 3,
+ 4,
+ tzinfo=pytz.FixedOffset(-450),
+ )
+ ),
+ "enabled": False,
+ }
+ ],
+ "colnames": ["event_time", "enabled"],
+ "coltypes": [
+ GenericDataType.TEMPORAL,
+ GenericDataType.BOOLEAN,
+ ],
+ "rowcount": 1,
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 13}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert [column["data_type"] for column in payload["columns"]] == [
+ "temporal",
+ "boolean",
+ ]
+ assert payload["data"] == [
+ {"event_time": "2024-01-01T02:03:04-07:30", "enabled": False}
+ ]
+
+
+@pytest.mark.asyncio
+async def test_saved_bullet_get_data_uses_strict_render_model(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=10,
+ slice_name="Strict Bullet",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "bullet", "metric": "Revenue"}',
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {"data": [{"Revenue": "not numeric"}], "colnames": ["Revenue"]}
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 10}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["error_type"] == "MalformedBulletOutput"
+
+
+@pytest.mark.asyncio
+async def test_saved_bullet_get_data_projects_dataframe_timestamps_to_epoch(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ from superset.dataframe import df_to_records
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ dateutil_dublin = dateutil_tz.gettz("Europe/Dublin")
+ dateutil_new_york = dateutil_tz.gettz("America/New_York")
+ assert dateutil_dublin is not None
+ assert dateutil_new_york is not None
+ dublin_fold = datetime(
+ 2024,
+ 10,
+ 27,
+ 1,
+ 30,
+ 0,
+ 123456,
+ tzinfo=dateutil_dublin,
+ fold=1,
+ )
+ new_york_gap = datetime(2024, 3, 10, 2, 30, 0, 123456, tzinfo=dateutil_new_york)
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=20,
+ slice_name="Timestamp Bullet",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(form_data),
+ )
+ source_values = [
+ pd.Timestamp("2024-01-02 03:04:05.123456789"),
+ pd.Timestamp("2024-01-02 08:34:05.123456789+05:30"),
+ pd.Timestamp(
+ datetime(
+ 2024,
+ 11,
+ 3,
+ 1,
+ 30,
+ tzinfo=ZoneInfo("America/New_York"),
+ fold=0,
+ )
+ ),
+ pd.Timestamp(
+ datetime(
+ 2024,
+ 11,
+ 3,
+ 1,
+ 30,
+ tzinfo=ZoneInfo("America/New_York"),
+ fold=1,
+ )
+ ),
+ pd.Timestamp("1969-12-31 23:59:59.999999999"),
+ pd.NaT,
+ dublin_fold,
+ new_york_gap,
+ datetime(2040, 7, 1, 12, 0, 0, 123456, tzinfo=dateutil_new_york),
+ pd.Timestamp(dublin_fold),
+ pd.Timestamp(new_york_gap),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 20}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert [row["Category"] for row in payload["data"]] == [
+ 1704164645123.456,
+ 1704164645123.456,
+ 1730611800000.0,
+ 1730615400000.0,
+ -0.0010000000000287557,
+ None,
+ 1729989000123.456,
+ 1710052200123.456,
+ 2224774800123.456,
+ 1729989000123.456,
+ 1710052200123.456,
+ ]
+
+
+@pytest.mark.asyncio
+async def test_transitionless_dateutil_dataframe_reaches_fastmcp_data(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from dateutil.zoneinfo import get_zonefile_instance
+ from fastmcp import Client
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+ from superset.utils.json import json_int_dttm_ser
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ names = [
+ "UTC",
+ "GMT",
+ "Universal",
+ "Zulu",
+ "EST",
+ "HST",
+ "MST",
+ "Etc/GMT+1",
+ "Etc/GMT-2",
+ ]
+ values = []
+ for getter in (dateutil_tz.gettz, get_zonefile_instance().get):
+ for name in names:
+ timezone_value = getter(name)
+ assert timezone_value is not None
+ values.append(
+ datetime(2040, 7, 1, 12, 34, 56, 123456, tzinfo=timezone_value)
+ )
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(values, dtype=object),
+ "Revenue": range(1, len(values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+
+ class _ProducerContext:
+ result_type = ChartDataResultType.FULL
+
+ def get_payload(self, **_kwargs: Any) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ producer_result = ProducerChartDataCommand(
+ _ProducerContext() # type: ignore[arg-type]
+ ).run()
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=22,
+ slice_name="Transitionless timestamps",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(form_data),
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return producer_result
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 22}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["row_count"] == len(values)
+ assert payload["total_rows"] == len(values)
+ assert [row["Category"] for row in payload["data"]] == [
+ json_int_dttm_ser(value) for value in values
+ ]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ ("format_", "excel_engine"),
+ [
+ ("json", None),
+ ("csv", None),
+ ("excel", "openpyxl"),
+ ("excel", "xlsxwriter"),
+ ],
+)
+async def test_saved_cached_bullet_duration_producer_reaches_fastmcp_exports(
+ mcp_server: Any,
+ mock_auth: Any,
+ format_: str,
+ excel_engine: str | None,
+) -> None:
+ import base64
+ import csv
+ import io
+ from contextlib import ExitStack
+ from unittest.mock import patch
+
+ from fastmcp import Client
+ from openpyxl import load_workbook
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ source_values = [
+ timedelta(0),
+ timedelta(microseconds=-1),
+ timedelta(days=1, seconds=2, microseconds=3),
+ pd.Timedelta(-1, unit="ns"),
+ pd.Timedelta("1 days 00:00:02.000003004"),
+ np.timedelta64(123456789, "ns"),
+ np.timedelta64("NaT"),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Duration": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+ assert type(rows[5]["Duration"]) is pd.Timedelta
+ assert rows[6]["Duration"] is None
+ expected = [
+ None
+ if row["Duration"] is None
+ else json.loads(json.dumps(row["Duration"], default=json.json_int_dttm_ser))
+ for row in rows
+ ]
+
+ class _ProducerContext:
+ result_type = ChartDataResultType.FULL
+
+ def get_payload(self, **_kwargs: Any) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Duration", "Revenue"],
+ "rowcount": len(rows),
+ "is_cached": True,
+ "cache_key": "duration-cache",
+ "cached_dttm": "2026-09-03T00:00:00+00:00",
+ }
+ ]
+ }
+
+ producer_result = ProducerChartDataCommand(
+ _ProducerContext() # type: ignore[arg-type]
+ ).run()
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Duration"],
+ }
+ chart = SimpleNamespace(
+ id=23,
+ slice_name="Duration Bullet",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(form_data),
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return producer_result
+
+ with ExitStack() as stack:
+ stack.enter_context(
+ patch.object(module, "find_chart_by_identifier", return_value=chart)
+ )
+ stack.enter_context(
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ )
+ )
+ stack.enter_context(
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ )
+ )
+ stack.enter_context(patch.object(command_module, "ChartDataCommand", _Command))
+ if excel_engine == "xlsxwriter":
+ stack.enter_context(
+ patch.object(
+ module, "_create_excel_with_openpyxl", side_effect=ImportError
+ )
+ )
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {"request": {"identifier": 23, "format": format_}},
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["row_count"] == len(rows)
+ assert payload["total_rows"] == len(rows)
+ assert payload["cache_status"]["cache_hit"] is True
+ if format_ == "json":
+ assert [row["Duration"] for row in payload["data"]] == expected
+ elif format_ == "csv":
+ decoded = list(csv.DictReader(io.StringIO(payload["csv_data"])))
+ assert [row["Duration"] or None for row in decoded] == expected
+ else:
+ workbook = load_workbook(io.BytesIO(base64.b64decode(payload["excel_data"])))
+ assert [cell.value for cell in workbook.active[1]] == ["Duration", "Revenue"]
+ assert [
+ workbook.active.cell(row=index + 2, column=1).value
+ for index in range(len(expected))
+ ] == expected
+
+
+@pytest.mark.asyncio
+async def test_form_data_key_bullet_duration_producer_uses_chart_data_wire(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Duration": pd.Series(
+ [timedelta(days=1, microseconds=3), pd.Timedelta(-1, "ns")],
+ dtype=object,
+ ),
+ "Revenue": [1, 2],
+ }
+ ),
+ convert_big_integers=False,
+ )
+ expected = [
+ json.loads(json.dumps(row["Duration"], default=json.json_int_dttm_ser))
+ for row in rows
+ ]
+
+ class _ProducerContext:
+ result_type = ChartDataResultType.FULL
+
+ def get_payload(self, **_kwargs: Any) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Duration", "Revenue"],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ producer_result = ProducerChartDataCommand(
+ _ProducerContext() # type: ignore[arg-type]
+ ).run()
+ form_data = {
+ "datasource": "1__table",
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Duration"],
+ }
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return producer_result
+
+ with (
+ patch.object(
+ module, "get_cached_form_data", return_value=json.dumps(form_data)
+ ),
+ patch.object(
+ module,
+ "build_query_context_from_form_data",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {"request": {"form_data_key": "duration-bullet"}},
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["chart_id"] == 0
+ assert [row["Duration"] for row in payload["data"]] == expected
- def run(self) -> dict[str, Any]:
- return {
- "queries": [
- {
- "colnames": ["primary"],
- "data": [],
- "rowcount": 0,
- },
- {
- "colnames": ["secondary"],
- "data": [{"secondary": 2}],
- "rowcount": 1,
- },
- ]
+
+GANTT_FORM_DATA: dict[str, Any] = {
+ "viz_type": "gantt_chart",
+ "start_time": "start_time",
+ "end_time": "end_time",
+ "y_axis": "task",
+}
+
+
+@pytest.mark.asyncio
+async def test_saved_empty_gantt_get_data_reports_no_data(
+ mcp_server: Any, mock_auth: Any
+) -> None:
+ """Only Bullet treats a zero-row result as renderable; Gantt reports NoData."""
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=22,
+ slice_name="Empty Gantt",
+ viz_type="gantt_chart",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(GANTT_FORM_DATA),
+ )
+ command = MagicMock()
+ command.run.return_value = {
+ "queries": [
+ {
+ "data": [],
+ "colnames": ["start_time", "end_time", "task"],
+ "rowcount": 0,
}
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", lambda _context: command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 22}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["error_type"] == "NoData"
+
+
+@pytest.mark.asyncio
+async def test_unsaved_empty_gantt_get_data_reports_no_data(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ from unittest.mock import AsyncMock
+ chart_data_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_data"
+ )
+ get_data_command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ command = MagicMock()
+ command.run.return_value = {
+ "queries": [{"data": [], "colnames": ["start_time", "end_time", "task"]}]
+ }
monkeypatch.setattr(
chart_data_module,
"build_query_context_from_form_data",
- lambda *_args, **_kwargs: object(),
+ lambda *_args, **_kwargs: _query_context_stub(),
)
monkeypatch.setattr(
- get_data_command_module, "ChartDataCommand", MultiQueryChartDataCommand
+ chart_data_module,
+ "event_logger",
+ SimpleNamespace(log_context=lambda **_kwargs: nullcontext()),
)
- request = GetChartDataRequest(form_data_key="mixed-chart")
+ monkeypatch.setattr(
+ get_data_command_module, "ChartDataCommand", lambda _context: command
+ )
+
response = await _query_from_form_data(
- {
- "datasource_id": 1,
- "datasource_type": "table",
- "viz_type": "mixed_timeseries",
- "row_limit": 10,
- },
- request,
+ {**GANTT_FORM_DATA, "datasource_id": 1, "datasource_type": "table"},
+ GetChartDataRequest(form_data_key="empty-gantt"),
AsyncMock(),
)
- assert isinstance(response, ChartData)
- assert response.data == []
- assert response.query_results is not None
- assert [result.data for result in response.query_results] == [
- [],
- [{"secondary": 2}],
- ]
+ assert isinstance(response, ChartError)
+ assert response.error_type == "NoData"
-def _make_chart_data(**overrides: Any) -> ChartData:
- """Build a minimal valid ChartData for testing."""
- from superset.mcp_service.common.cache_schemas import CacheStatus
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ ("grouped", "format_", "identifier_alias", "excel_engine"),
+ [
+ (False, "json", "id", None),
+ (True, "json", "chart_id", None),
+ (False, "csv", "id", None),
+ (True, "csv", "chart_id", None),
+ (False, "excel", "id", "openpyxl"),
+ (True, "excel", "id", "openpyxl"),
+ (False, "excel", "chart_id", "xlsxwriter"),
+ (True, "excel", "chart_id", "xlsxwriter"),
+ ],
+)
+async def test_saved_empty_bullet_get_data_fastmcp_returns_normalized_rows(
+ mcp_server: Any,
+ mock_auth: Any,
+ grouped: bool,
+ format_: str,
+ identifier_alias: str,
+ excel_engine: str | None,
+) -> None:
+ import base64
+ import io
+ from contextlib import ExitStack
+ from unittest.mock import patch
- defaults: dict[str, Any] = {
- "chart_id": 1,
- "chart_name": "Test Chart",
- "chart_type": "table",
- "columns": [],
- "data": [{"metric": 3.5}],
- "row_count": 1,
- "total_rows": 1,
- "data_freshness": None,
- "summary": "summary",
- "insights": [],
- "data_quality": {},
- "recommended_visualizations": [],
- "performance": PerformanceMetadata(
- query_duration_ms=42,
- cache_status="fresh_query",
- ),
- "cache_status": CacheStatus(cache_hit=False),
+ from fastmcp import Client
+ from openpyxl import load_workbook
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ form_data: dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
}
- defaults.update(overrides)
- return ChartData(**defaults)
+ if grouped:
+ form_data["groupby"] = ["Region"]
+ raw_columns = ["Revenue", *(["Region"] if grouped else [])]
+ chart = SimpleNamespace(
+ id=21,
+ slice_name="Empty Bullet",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(form_data),
+ )
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
-class TestChartDataTotalRowsCoercion:
- """Regression tests: float total_rows causes PydanticSerializationError."""
+ def validate(self) -> None: ...
- def test_float_total_rows_is_coerced_to_int(self) -> None:
- chart_data = _make_chart_data(total_rows=5.0)
- assert chart_data.total_rows == 5
- assert isinstance(chart_data.total_rows, int)
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": 2
+ * [
+ {
+ "data": [],
+ "colnames": raw_columns,
+ "rowcount": 0,
+ }
+ ]
+ }
+
+ with ExitStack() as stack:
+ stack.enter_context(
+ patch.object(module, "find_chart_by_identifier", return_value=chart)
+ )
+ stack.enter_context(
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ )
+ )
+ stack.enter_context(
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ )
+ )
+ stack.enter_context(patch.object(command_module, "ChartDataCommand", _Command))
+ if excel_engine == "xlsxwriter":
+ stack.enter_context(
+ patch.object(
+ module, "_create_excel_with_openpyxl", side_effect=ImportError
+ )
+ )
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {
+ "request": {
+ identifier_alias: 21,
+ "format": format_,
+ }
+ },
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert "error_type" not in payload
+ if format_ == "json":
+ assert payload["data"] == []
+ assert payload["row_count"] == 0
+ assert payload["total_rows"] == 0
+ assert len(payload["query_results"]) == 2
+ for query_result in payload["query_results"]:
+ assert query_result["data"] == []
+ assert query_result["row_count"] == 0
+ assert query_result["total_rows"] == 0
+ elif format_ == "csv":
+ assert payload["format"] == "csv"
+ assert payload["row_count"] == 0
+ assert payload["total_rows"] == 0
+ assert payload["csv_data"] == (
+ "Revenue,Region\r\n" if grouped else "Revenue\r\n"
+ )
+ else:
+ assert payload["format"] == "excel"
+ assert payload["row_count"] == 0
+ assert payload["total_rows"] == 0
+ workbook = load_workbook(io.BytesIO(base64.b64decode(payload["excel_data"])))
+ assert [cell.value for cell in workbook.active[1]] == raw_columns
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ "grouped",
+ [
+ False,
+ True,
+ ],
+)
+async def test_form_data_key_empty_bullet_fastmcp_returns_normalized_rows(
+ mcp_server: Any,
+ mock_auth: Any,
+ grouped: bool,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ form_data: dict[str, Any] = {
+ "datasource": "1__table",
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ }
+ if grouped:
+ form_data["groupby"] = ["Region"]
+ raw_columns = ["Revenue", *(["Region"] if grouped else [])]
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": 2
+ * [
+ {
+ "data": [],
+ "colnames": raw_columns,
+ "rowcount": 0,
+ }
+ ]
+ }
+
+ with (
+ patch.object(
+ module, "get_cached_form_data", return_value=json.dumps(form_data)
+ ),
+ patch.object(
+ module,
+ "build_query_context_from_form_data",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {
+ "request": {
+ "form_data_key": "empty-bullet",
+ "format": "json",
+ }
+ },
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert "error_type" not in payload
+ assert payload["chart_id"] == 0
+ assert payload["chart_type"] == "bullet"
+ assert payload["row_count"] == 0
+ assert payload["total_rows"] == 0
+ assert payload["data"] == []
+ assert len(payload["query_results"]) == 2
+ for query_result in payload["query_results"]:
+ assert query_result["data"] == []
+ assert query_result["row_count"] == 0
+ assert query_result["total_rows"] == 0
def test_float_total_rows_serializes_without_error(self) -> None:
Review Comment:
These four total_rows tests are now local functions inside the new async Bullet test, so pytest no longer collects the float-serialization, nullable, integer and fractional-value regressions. Could they be moved back into TestChartDataTotalRowsCoercion, or made module-level tests without self?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4183996396
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,464 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ temporal_pairs: list[tuple[str, str]] = []
+ inert_subjects: set[str] = set()
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ if not isinstance(comparator, str) or not comparator:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs a range"
+ )
+ if comparator.casefold() == "no filter":
+ inert_subjects.add(subject)
+ else:
+ temporal_pairs.append((subject, comparator))
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ if len(temporal_pairs) > 1:
+ raise ValueError(
+ "Multiple active native temporal filters cannot be represented by "
+ "a single temporal_column/time_range pair"
+ )
+ if temporal_pairs:
+ subject, comparator = temporal_pairs[0]
+ if data.get("temporal_column") not in (None, subject) or data.get(
+ "time_range"
+ ) not in (None, "No filter", comparator):
+ raise ValueError(
+ "Native temporal filter conflicts with temporal_column/time_range"
+ )
+ data["temporal_column"] = subject
+ data["time_range"] = comparator
+ elif len(inert_subjects) == 1:
+ data.setdefault("temporal_column", next(iter(inert_subjects)))
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ from superset.mcp_service.chart.preview_utils import (
+ _bullet_numeric_control_tokens,
+ )
+
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ # Saved Explore controls are lenient; newly authored typed lists
+ # still undergo the strict List[float] validation below.
+ for control in ("ranges", "markers", "marker_lines"):
+ if control in data:
+ data[control] = _bullet_numeric_control_tokens(
+ data[control], control
+ )
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
Review Comment:
Fixed in 2831272d9b3c33d67f25671bc0dc22e84f657bfd; the order_by case is now covered too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4137208372
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1839 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Integer/Decimal bounds also prevent
+# later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget
+) -> str | None:
+ """Bound exact-container query metadata independently of row data."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = list.__len__(item)
+ if width > _MAX_ROW_CONTAINER_ITEMS:
Review Comment:
This caps `indexnames` at 4,096 items along with every other metadata array, but `indexnames` carries one entry per row (it's `list(df.index)`), the same cardinality as `data`, which this file otherwise allows up to `MAX_QUERY_RESULT_ROWS = 50_000`. So a plain query returning, say, 5,000 rows would come back as `MalformedQueryResult` instead of its actual data. Should row-shaped metadata fields like `indexnames` be exempted from this generic array cap, or bounded against the row limit instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4240826317
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -3489,7 +3521,655 @@ def _as_column_list(value: Any) -> list[Any]:
return [value]
-def analyze_chart_capabilities(viz_type: str | None, config: Any) -> ChartCapabilities:
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ # A missing saved viz_type is a boundary too, matching merge_chart_form_data.
+ if existing_form_data.get("viz_type") != new_form_data.get("viz_type"):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+ if (
+ existing_binding is None
+ and existing_subject is None
+ and range_explicit
+ and not subject_authoritative
+ and "filters" not in explicit_fields
+ ):
+ # A range-only update on an unmarked chart adopts its saved temporal
+ # filter even when Explore bound it to a column other than the mapper's
+ # proposed subject. Zero or several native ranges stay ambiguous, so the
+ # proposed binding is added instead of guessing which one to rewrite.
+ unmarked_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("expressionType") == "SIMPLE"
+ and filter_.get("clause") == "WHERE"
+ and isinstance(filter_.get("subject"), str)
+ ]
+ if len(unmarked_matches) == 1:
+ existing_binding = unmarked_matches[0]
+ existing_subject = existing_binding["subject"]
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
Review Comment:
Valid, fixed in 6c28acc6da586e89417493be58f4f090efe18e27. With explicit filters, an explicit time_range and no saved binding to rewrite, the merge now falls back to the mapper's generated binding instead of dropping it, so region=North plus Last month saves both filters. Regression test: test_bullet_explicit_filters_and_range_without_saved_binding_keep_range.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1809,7 +1822,1063 @@ def generate_xy_vega_lite_preview(
spec["transform"] = [
{"fold": series_columns, "as": [series_field, value_field]}
]
- encoding["y"] = {"field": value_field, "type": "quantitative"}
+ # Vega-Lite stacks bar and area marks colored by a nominal field
+ # by default; follow the chart's saved ``stack`` control instead.
+ encoding["y"] = {
+ "field": value_field,
+ "type": "quantitative",
+ "stack": _xy_vega_lite_stack(form_data.get("stack")),
+ }
encoding["color"] = {"field": series_field, "type": "nominal"}
+ if (
+ spec.get("mark") == "bar"
+ and encoding["y"]["stack"] is None
+ and "x" in encoding
+ ):
+ # Unstacked bars sit side by side like Explore's grouped bars.
+ # Offset channels need a discrete band x scale, so temporal
+ # values stay date-formatted through a millisecond time unit so
+ # sub-second timestamps keep distinct bands as in Explore.
+ x_encoding = encoding["x"]
+ if x_encoding.get("type") == "temporal":
+ x_encoding["timeUnit"] = (
+ "yearmonthdatehoursminutessecondsmilliseconds"
+ )
+ x_encoding["type"] = "ordinal"
+ encoding["xOffset"] = {"field": series_field, "type": "nominal"}
return preview
+
+
+def _xy_vega_lite_stack(stack: Any) -> str | None:
+ """Map the ECharts timeseries ``stack`` control to a Vega-Lite stack."""
+ if not stack:
+ return None
+ return "normalize" if stack == "Expand" else "zero"
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+_MAX_BULLET_FIELDS = 256
+
+
+_MAX_BULLET_FIELD_BYTES = 1000
+
+
+_MAX_BULLET_TEXT_BYTES = 2000
+
+
+# ECMAScript WhiteSpace and LineTerminator characters used by trim/Number.
+_JAVASCRIPT_WHITESPACE = (
+ "\t\n\v\f\r \u00a0\u1680\u2000\u2001\u2002\u2003\u2004"
+ "\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000\ufeff"
+)
+
+
+_MAX_BULLET_TOKENS = 256
+
+
+# A comma-joined presentation control holds up to _MAX_BULLET_TOKENS tokens of
+# _MAX_BULLET_TEXT_BYTES each, so the whole-control bound must cover that many
+# tokens plus separators rather than reuse the per-token bound.
+_MAX_BULLET_CONTROL_CHARS = _MAX_BULLET_TOKENS * (_MAX_BULLET_TEXT_BYTES + 1)
+
+
+_ENUM_SCALAR_TYPES = (str, int, float, bool, Decimal)
+
+
+class BulletOutputError(ValueError):
+ """A Bullet query result cannot be rendered without guessing its roles."""
+
+ def __init__(self, message: str, error_type: str = "MalformedBulletOutput") -> None:
+ super().__init__(message)
+ self.error_type = error_type
+
+
+@dataclass(frozen=True)
+class BulletRenderModel:
+ """Strict, frontend-aligned data and presentation roles for one preview."""
+
+ rows: list[dict[str, Any]]
+ metric_field: str
+ dimensions: list[str]
+ measures: list[float]
+ ranges: list[float]
+ range_labels: list[str]
+ markers: list[float]
+ marker_labels: list[str]
+ marker_lines: list[float]
+ marker_line_labels: list[str]
+ y_axis_format: str
+ show_labels: bool
+ show_legend: bool
+
+
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ # getColumnLabel: an adhoc column without a label is keyed by its SQL.
+ for key in ("label", "sqlExpression", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if type(value) in {list, tuple, dict}:
+ data, failure = first_query_data(
+ {"queries": [{"data": [{"value": value}]}]}, temporal_json_numbers=True
+ )
+ if failure is not None or data is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} "
+ "has an invalid or unbounded container value"
+ )
+ normalized = data[0]["value"]
+ reason = None
+ elif _is_chart_data_temporal_scalar(value := _safe_enum_backing(value)):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_scalar(value)
+ if (
+ reason is None
+ and normalized is None
+ and value is not None
+ and type(value) in {float, np.float16, np.float32, np.float64}
+ and math.isinf(float(value))
+ ):
+ # NaN keeps the shared null normalization, but an infinite source
+ # value has no faithful category text and is rejected.
+ reason = "contains a non-finite number"
+ if (
+ reason is None
+ and type(normalized) is str
+ and _bounded_utf8_length(normalized, _MAX_BULLET_TEXT_BYTES) is None
Review Comment:
Valid, fixed in 6c28acc6da586e89417493be58f4f090efe18e27. resolve_bullet_render_model and _bullet_category_value take bound_labels; the raw-data paths (normalize_query_result and sanitize_data_rows, used by get_chart_data and JSON/CSV/Excel exports) pass False, so dimension strings are checked against the shared MAX_RESULT_STRING_LENGTH and no label text is derived. The ASCII and Vega preview renderers still apply the 2,000-byte label bound. Regression test: test_bullet_raw_rows_keep_shared_limit_for_long_dimension_text.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4225421693
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,120 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list, _time_comparison
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ def vega_field(name: str) -> str:
+ """Escape a literal key so Vega-Lite does not read it as a nested path."""
+ return "".join("\\" + char if char in ".[]\\" else char for char in name)
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ series_labels = list(metric_labels)
+ if len(metric_labels) == 1 and _time_comparison(
+ form_data, _as_list(form_data.get("metrics"))
+ ):
+ # A single compared metric renames each shifted series to its bare
+ # offset, so those series carry the offset rather than the metric.
+ series_labels.extend(
+ str(offset) for offset in _as_list(form_data.get("time_compare"))
+ )
+ prefixes = {
+ spelling
+ for label in series_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(prefix + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(prefix + "__")
+ for prefix in prefixes
+ )
+ ]
+ if not fields and len(metric_labels) == 1 and form_data.get("truncate_metric"):
+ # A single truncated metric drops its label from the pivoted column
+ # names, so every non-x-axis column is one category series.
+ fields = [field for field in data[0] if field != x_axis]
+ if not fields:
+ return None
+ sample = data[0][x_axis]
+ x_type = (
+ "temporal"
+ if isinstance(sample, str) and any(char in sample for char in "-/: ")
Review Comment:
Adding a space to this character set makes any grouped category label containing a space (`New York`, `Product A`) a temporal x-axis, which Vega-Lite parses as an invalid date, so a bar chart by `product` and `region` previews with no bars while Explore renders it. ISO dates and timestamps already contain `-` or `:`, so the space only adds false positives. Can the x type come from the axis/column type instead of scanning the first value?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,853 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if isinstance(existing_viz_type, str) and existing_viz_type != new_form_data.get(
Review Comment:
A saved chart whose `params` omit `viz_type` (REST-created charts can; `Slice.form_data` injects it from the column) is treated as a viz change by `merge_chart_form_data`, which returns the fresh form data, but this guard only returns for a string `viz_type`, so the old `adhoc_filters` are restored. Converting a Pie chart saved with a `region = 'US'` filter to a Table with no `filters` then returns only US rows, and `BulletChartPlugin.resolve_update_config` likewise ignores the saved hierarchy for such a chart. Should `_get_existing_form_data` inject `chart.viz_type` (as `Slice.form_data` does) so both merges agree, or should this guard treat a missing value like a mismatch the way `merge_same_viz_form_data` does?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -589,23 +591,518 @@ def resolve_big_number_columns(form_data: dict[str, Any]) -> list[Any]:
return [granularity] if isinstance(granularity, str) and granularity else []
-def resolve_gantt_query_fields( # noqa: C901
- form_data: dict[str, Any],
-) -> tuple[list[Any], list[Any], list[list[Any]], list[Any]]:
- """Mirror the ECharts Gantt ``buildQuery`` field extraction contract.
+def extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
+ """Return the x_axis column name from form_data, or None if not set."""
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis:
+ return x_axis
+ if isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ return col_name if isinstance(col_name, str) and col_name else None
+ return None
+
+
+def _x_axis_query_field(form_data: dict[str, Any]) -> Any | None:
+ """Resolve a frontend x-axis value without losing SQL expressions."""
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis:
+ return x_axis
+ if isinstance(x_axis, dict):
+ if (
+ isinstance(x_axis.get("sqlExpression"), str)
+ and x_axis.get("sqlExpression")
+ and isinstance(x_axis.get("label"), str)
+ and x_axis.get("label")
+ and x_axis.get("expressionType") in (None, "SQL")
+ ):
+ return x_axis
+ column_name = x_axis.get("column_name") or x_axis.get("columnName")
+ if isinstance(column_name, str) and column_name:
+ return column_name
+ return None
+
+
+def _normalized_x_axis_query_field(form_data: dict[str, Any]) -> Any | None:
+ """Mirror ``buildQueryContext.normalizeTimeColumn`` for a set x-axis."""
+ x_axis = _x_axis_query_field(form_data)
+ if x_axis is None:
+ return None
+ time_grain = form_data.get("time_grain_sqla")
+ if isinstance(x_axis, str):
+ normalized = {
+ "columnType": "BASE_AXIS",
+ "sqlExpression": x_axis,
+ "label": x_axis,
+ "expressionType": "SQL",
+ "isColumnReference": True,
+ }
+ if time_grain is not None:
+ normalized["timeGrain"] = time_grain
+ return normalized
+ normalized = {"columnType": "BASE_AXIS", **x_axis}
+ # The original adhoc column's grain overrides the common control, matching
+ # the frontend spread order.
+ if "timeGrain" not in normalized and time_grain is not None:
+ normalized["timeGrain"] = time_grain
+ return normalized
+
+
+def _resolve_big_number_query_columns(form_data: dict[str, Any]) -> list[Any]:
+ """Resolve only Big Number's explicit x-axis query column.
+
+ The frontend keeps ``granularity_sqla`` out of ``columns`` and asks the
+ backend for a timeseries instead, which yields ``__timestamp``. An explicit
+ ``x_axis`` is different: the plugin retains that column in the final query.
+ """
+ if (x_axis := _normalized_x_axis_query_field(form_data)) is not None:
+ return [x_axis]
+ return []
+
+
+def _as_list(value: Any) -> list[Any]:
+ """Match the frontend's ``ensureIsArray`` for query controls."""
+ if value is None:
+ return []
+ return value if isinstance(value, list) else [value]
+
+
+def _column_label(column: Any) -> str | None:
+ """Return the frontend ``getColumnLabel`` value for a query column."""
+ if isinstance(column, str):
+ return column
+ if not isinstance(column, dict):
+ return None
+ return (
+ column.get("label") or column.get("sqlExpression") or column.get("column_name")
+ )
+
+
+def _metric_label(metric: Any) -> str | None:
+ """Return the frontend ``getMetricLabel`` value for a query metric."""
+ if isinstance(metric, str):
+ return metric
+ if not isinstance(metric, dict):
+ return None
+ if label := metric.get("label"):
+ return label
+ if metric.get("expressionType") == "SIMPLE":
+ column = metric.get("column") or {}
+ name = (
+ column.get("columnName") or column.get("column_name")
+ if isinstance(column, dict)
+ else None
+ )
+ if name and metric.get("aggregate"):
+ return f"{metric['aggregate']}({name})"
+ return metric.get("sqlExpression")
+
+
+def _is_query_form_metric(value: Any) -> bool:
+ """Mirror the frontend's ``isQueryFormMetric`` type guard."""
+ return isinstance(value, str) or (
+ isinstance(value, dict) and value.get("expressionType") in {"SIMPLE", "SQL"}
+ )
- Returns ``(columns, metrics, orderby, series_columns)``. Saved form data is
- user-editable, so malformed or oversized native ordering is rejected rather
- than silently dropped or passed into ``QueryContextFactory``.
+
+def _timeseries_base_metrics(form_data: dict[str, Any]) -> list[Any]:
+ """Return metrics extracted by the common frontend query-field aliases."""
+ metrics = [*_as_list(form_data.get("metrics")), *_as_list(form_data.get("metric"))]
+ if (size := form_data.get("size")) is not None:
+ metrics.append(size)
+ return _dedupe_query_fields(metrics, _metric_label)
+
+
+def _timeseries_extra_metrics(form_data: dict[str, Any]) -> list[Any]:
+ """Mirror ``extractExtraMetrics`` for ungrouped x-axis sorting."""
+ if _as_list(form_data.get("groupby")):
+ return []
+ limit_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if not limit_metrics:
+ return []
+ limit_metric = limit_metrics[0]
+ limit_label = _metric_label(limit_metric)
+ if not limit_label or limit_label != form_data.get("x_axis_sort"):
+ return []
+ if any(
+ _metric_label(metric) == form_data.get("x_axis_sort")
+ for metric in _as_list(form_data.get("metrics"))
+ ):
+ return []
+ return [limit_metric]
+
+
+def _query_series_columns(query: dict[str, Any]) -> list[Any]:
+ """Resolve pivot columns with JavaScript's array-truthiness semantics.
+
+ JavaScript treats an explicitly empty array as truthy, so
+ ``series_columns: []`` must not fall through to the query's x-axis column.
+ Only an absent/null series-columns field falls back to ``columns``.
"""
- from superset.utils import json as utils_json
+ if "series_columns" in query and query["series_columns"] is not None:
+ return _as_list(query["series_columns"])
+ return _as_list(query.get("columns"))
+
+
+def _dedupe_query_fields(values: list[Any], labeler: Any) -> list[Any]:
+ """Preserve the first query field for each frontend result label."""
+ result: list[Any] = []
+ seen: set[str] = set()
+ for value in values:
+ label = labeler(value)
+ if not label or label in seen:
+ continue
+ seen.add(label)
+ result.append(value)
+ return result
+
+
+def _deck_tooltip_columns(value: Any) -> list[str]:
+ """Extract the physical tooltip fields accepted by Deck.gl plugins."""
+ if not isinstance(value, list):
+ return []
+ columns: list[str] = []
+ for item in value:
+ column: Any = None
+ if isinstance(item, str):
+ column = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ column = item.get("column_name")
+ if isinstance(column, str) and column and column not in columns:
+ columns.append(column)
+ return columns
+
+
+def _deck_base_query_fields( # noqa: C901
+ form_data: dict[str, Any],
+) -> tuple[list[Any], list[Any], list[list[Any]] | None]:
+ """Mirror the common fields built before a Deck.gl layer adapter runs."""
+ query_mode = form_data.get("query_mode")
+ columns: list[Any] = []
+ metrics: list[Any] = []
+ raw_orderby: list[Any] = []
+ aliases = {
+ "metric": "metrics",
+ "metric_2": "metrics",
+ "secondary_metric": "metrics",
+ "left_metric": "metrics",
+ "right_metric": "metrics",
+ "x": "metrics",
+ "y": "metrics",
+ "size": "metrics",
+ "all_columns": "columns",
+ "series": "groupby",
+ "order_by_cols": "orderby",
+ }
+ for key, value in form_data.items():
+ if value is None:
+ continue
+ normalized = aliases.get(key, key)
+ if query_mode == "aggregate" and normalized == "columns":
+ continue
+ if query_mode == "raw" and normalized in {"groupby", "metrics"}:
+ continue
+ if normalized == "groupby":
+ normalized = "columns"
+ if normalized == "columns":
+ columns.extend(_as_list(value))
+ elif normalized == "metrics":
+ metrics.extend(_as_list(value))
+ elif normalized == "orderby":
+ raw_orderby.extend(_as_list(value))
+
+ orderby: list[list[Any]] = []
+ if len(raw_orderby) > 100:
+ raise ValueError("Deck orderby must contain at most 100 entries")
+ for index, value in enumerate(raw_orderby):
+ if isinstance(value, str):
+ if len(value) > 1000:
+ raise ValueError(f"Deck orderby[{index}] is too long")
+ try:
+ value = utils_json.loads(value)
+ except (TypeError, ValueError) as ex:
+ raise ValueError(f"Deck orderby[{index}] is not valid JSON") from ex
+ if (
+ not isinstance(value, (list, tuple))
+ or len(value) != 2
+ or not isinstance(value[1], bool)
+ ):
+ raise ValueError(
+ f"Deck orderby[{index}] must be [field, ascending_boolean]"
+ )
+ orderby.append(list(value))
+
+ return (
+ _dedupe_query_fields(columns, _column_label),
+ _dedupe_query_fields(metrics, _metric_label),
+ orderby or None,
+ )
+
+
+def _deck_add_columns(columns: list[Any], *values: Any) -> list[Any]:
+ """Add layer fields by frontend result label without duplicates."""
+ expanded = list(columns)
+ for value in values:
+ expanded.extend(_as_list(value))
+ return _dedupe_query_fields(expanded, _column_label)
+
+
+def _deck_add_metrics(metrics: list[Any], *values: Any) -> list[Any]:
+ """Add layer metric roles without emitting the same output label twice."""
+ expanded = list(metrics)
+ for value in values:
+ expanded.extend(_as_list(value))
+ return _dedupe_query_fields(expanded, _metric_label)
+
+
+def _deck_fixed_or_metric(
+ value: Any, *, allow_legacy_string: bool = True
+) -> Any | None:
+ """Return the metric stored in a Deck fixed-or-metric control."""
+ if allow_legacy_string and isinstance(value, str) and value:
+ # Legacy Deck controls store the metric key directly as a string.
+ return value
+ if not isinstance(value, dict) or value.get("type") != "metric":
+ return None
+ metric = value.get("value")
+ return (
+ metric if metric is not None and not isinstance(metric, (int, float)) else None
+ )
+
+
+def _deck_spatial_columns(value: Any) -> list[str]:
+ """Resolve one complete frontend Deck spatial configuration."""
+ if not isinstance(value, dict) or not value.get("type"):
+ raise ValueError("Bad spatial key")
+ spatial_type = value["type"]
+ fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if fields is None:
+ raise ValueError(f"Unknown spatial type: {spatial_type}")
+ columns: list[str] = []
+ for field in fields:
+ column = value.get(field)
+ if not isinstance(column, str) or not column:
+ raise ValueError(f"Incomplete {spatial_type} spatial configuration")
+ columns.append(column)
+ return columns
+
+
+def _deck_add_null_filters(
+ query: dict[str, Any], columns: list[str], *, include_null_value: bool = False
+) -> None:
+ """Append renderer-required non-null filters, deduplicated by column."""
+ filters = list(query.get("filters") or [])
+ present = {
+ item.get("col")
+ for item in filters
+ if isinstance(item, dict) and item.get("op") == "IS NOT NULL"
+ }
+ for column in columns:
+ if not column or column in present:
+ continue
+ filter_: dict[str, Any] = {"col": column, "op": "IS NOT NULL"}
+ if include_null_value:
+ filter_["val"] = None
+ filters.append(filter_)
+ present.add(column)
+ query["filters"] = filters
+
+
+def _build_deck_query( # noqa: C901
Review Comment:
With Deck queries now built here, `resolve_deck_gl_columns`, `_resolve_deck_gl_metrics` and `_deck_gl_null_filters` (and their helpers `_deck_gl_spatial_cols` and `_is_metric_ref`) have no production caller, yet `test_chart_helpers.py` still exercises them. A later fix to Scatter tooltip-column selection could land in `resolve_deck_gl_columns`, pass its tests, and change no MCP query. Can they and their tests be removed?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,2066 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+_EXCEL_MIN_DATE = date(1900, 1, 1)
+# Leave room for XLSX readers' millisecond rounding at the final serial day.
+_EXCEL_MAX_TIME = time(23, 59, 59, 999000)
+_EXCEL_MAX_DATETIME = datetime.combine(date.max, _EXCEL_MAX_TIME)
+# Serial day 2958465 is 9999-12-31 in Excel's 1900 date system.
+_EXCEL_MAX_DURATION = timedelta(
+ days=2958465, hours=23, minutes=59, seconds=59, milliseconds=999
+)
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
+
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
Review Comment:
`df_to_records` nulls top-level infinities, but an infinity inside an array cell (say a Postgres `float8[]` holding `'Infinity'`) reaches this branch and fails the entire `query_dataset`/`get_table` result with `MalformedQueryResult`. The NaN case a few lines up already maps to `None`, so should a nested infinity be nulled the same way instead of rejecting every row?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,853 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if isinstance(existing_viz_type, str) and existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+def _currency_form_value(value: CurrencyFormat | None) -> dict[str, str] | None:
+ """Return the native value for an explicitly supplied currency control."""
+ return value.to_form_data() if value is not None else None
+
+
+def _column_names(value: Sequence[ColumnRef] | None) -> list[str | None] | None:
+ """Return a native column-name list while retaining an explicit null."""
+ return [column.name for column in value] if value is not None else None
+
+
+def _table_sort_value(value: Sequence[str | SortByConfig] | None) -> list[str] | None:
+ """Return the native Table sort control for an explicit typed value."""
+ if value is None:
+ return None
+ return [
+ json.dumps(
+ [entry.column, entry.ascending]
+ if isinstance(entry, SortByConfig)
+ else [entry, False]
+ )
+ for entry in value
+ ]
+
+
+def _table_column_config_value(value: Any) -> dict[str, Any] | None:
+ """Return Table column config without losing an explicit null or empty map."""
+ if value is None:
+ return None
+ return {
+ label: column.model_dump(by_alias=True, exclude_unset=True)
+ for label, column in value.items()
+ }
+
+
+# Mappers intentionally omit optional controls so fresh charts use the frontend
+# defaults. During a same-viz update, however, an explicitly supplied false,
+# null, or empty value must block preservation of the saved native key. Keep the
+# typed-to-native relationship declarative so every update path shares it.
+_FormValueConverter = Callable[[Any], Any]
+_FormControlMap = dict[str, tuple[str, _FormValueConverter]]
+
+_COMMON_EXPLICIT_FORM_CONTROLS: _FormControlMap = {
+ "color_scheme": ("color_scheme", lambda value: value),
+ "currency_format": ("currency_format", _currency_form_value),
+ "show_value": ("show_value", lambda value: value),
+}
+
+_CHART_EXPLICIT_FORM_CONTROLS: dict[str, _FormControlMap] = {
+ "table": {
+ "sort_by": ("order_by_cols", _table_sort_value),
+ "column_config": ("column_config", _table_column_config_value),
+ },
+ "xy": {
+ "group_by": ("groupby", _column_names),
+ "series_limit": ("series_limit", lambda value: value),
+ "stacked": ("stack", lambda value: "Stack" if value else None),
+ "orientation": ("orientation", lambda value: value),
+ "legend_orientation": ("legendOrientation", lambda value: value),
+ "x_axis_time_format": ("x_axis_time_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "mixed_timeseries": {
+ "group_by": ("groupby", _column_names),
+ "group_by_secondary": ("groupby_b", _column_names),
+ "currency_format_secondary": (
+ "currency_format_secondary",
+ _currency_form_value,
+ ),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "waterfall": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "big_number": {
+ "subheader": ("subheader", lambda value: value),
+ "y_axis_format": ("y_axis_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "compare_lag": ("compare_lag", lambda value: value),
+ "time_format": ("time_format", lambda value: value),
+ "aggregation": ("aggregation", lambda value: value),
+ },
+ "handlebars": {
+ "style_template": ("styleTemplate", lambda value: value),
+ "columns": ("all_columns", _column_names),
+ "groupby": ("groupby", _column_names),
+ "metrics": ("metrics", _column_names),
+ },
+ "pivot_table": {
+ "date_format": ("date_format", lambda value: value),
+ },
+ "interactive_pivot": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "series_limit": ("series_limit", lambda value: value),
+ "date_format": ("date_format", lambda value: value),
+ "column_sort": ("colOrder", lambda value: value),
+ },
+}
+
+
+def _apply_explicit_form_controls( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply typed controls whose mapper omission represents a native clear."""
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ controls = {
+ **_COMMON_EXPLICIT_FORM_CONTROLS,
+ **_CHART_EXPLICIT_FORM_CONTROLS.get(config.chart_type, {}),
+ }
+ for field_name, (native_key, convert) in controls.items():
+ if field_name in explicit_fields:
+ converted = convert(getattr(config, field_name))
+ is_clear = (
+ converted is None
+ or converted is False
+ or converted == ""
+ or converted in ([], {})
+ )
+ if is_clear:
+ new_form_data[native_key] = converted
+
+ axis_controls = {
+ "xy": (
+ ("x_axis", "x_axis_title", "x_axis_format", None),
+ ("y_axis", "y_axis_title", "y_axis_format", "logAxis"),
+ ),
+ "mixed_timeseries": (
+ ("x_axis", "xAxisTitle", "x_axis_time_format", None),
+ ("y_axis", "yAxisTitle", "y_axis_format", "logAxis"),
+ (
+ "y_axis_secondary",
+ "yAxisTitleSecondary",
+ "y_axis_format_secondary",
+ "logAxisSecondary",
+ ),
+ ),
+ }
+ if config.chart_type in axis_controls:
+ for field_name, title_key, format_key, scale_key in axis_controls[
+ config.chart_type
+ ]:
+ if field_name not in explicit_fields:
+ continue
+ axis = getattr(config, field_name)
+ if axis is None:
+ new_form_data[title_key] = None
+ new_form_data[format_key] = None
+ if scale_key:
+ new_form_data[scale_key] = None
+ continue
+ axis_fields = set(axis.model_fields_set)
+ if "title" in axis_fields:
+ new_form_data[title_key] = axis.title
+ if "format" in axis_fields:
+ new_form_data[format_key] = axis.format
+ if scale_key and "scale" in axis_fields:
+ new_form_data[scale_key] = (
+ None if axis.scale is None else axis.scale == "log"
+ )
+
+ if config.chart_type == "xy" and "legend" in explicit_fields:
+ legend = config.legend
+ if legend is None:
+ new_form_data["show_legend"] = None
+ new_form_data["legendOrientation"] = None
+ else:
+ legend_fields = set(legend.model_fields_set)
+ if "show" in legend_fields:
+ new_form_data["show_legend"] = legend.show
+ if "position" in legend_fields:
+ new_form_data["legendOrientation"] = legend.position
+
+ if config.chart_type == "interactive_pivot":
+ if "temporal_column" in explicit_fields and config.temporal_column is None:
+ new_form_data["granularity_sqla"] = None
+ new_form_data["temporal_columns_lookup"] = None
+ if (
+ "series_limit_metric" in explicit_fields
+ and config.series_limit_metric is None
+ ):
+ new_form_data["series_limit_metric"] = None
+ if "comparison_period" in explicit_fields and config.comparison_period is None:
+ new_form_data["time_compare"] = None
+ if "comparison_type" in explicit_fields and config.comparison_type is None:
+ new_form_data["comparison_type"] = None
+
+ # A Waterfall axis replacement cannot inherit a bucket belonging to the old
+ # temporal subject. Grain omission preserves only while the axis is stable;
+ # explicit null is already handled by the declarative control map above.
+ if (
+ config.chart_type == "waterfall"
+ and existing_form_data.get("x_axis") != new_form_data.get("x_axis")
+ and "time_grain" not in explicit_fields
+ ):
+ new_form_data["time_grain_sqla"] = None
+
+
+def merge_same_viz_form_data(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Preserve saved controls that the typed mapper does not represent.
+
+ The typed MCP surface deliberately models a bounded subset of every Explore
+ control panel. For a replacement within the exact same native ``viz_type``,
+ keys absent from the mapper therefore represent omitted controls and retain
+ their saved values. Mapper output and the chart-specific merge helpers run
+ first and remain authoritative, including explicit empty, false, null, and
+ nested values.
+
+ No generic state crosses a visualization boundary. This prevents query-role
+ keys from the previous plugin (for example ``metric`` or ``groupby``) from
+ leaking into a different plugin whose role contract is unrelated.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if not isinstance(existing_viz_type, str) or existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+
+ _apply_explicit_form_controls(existing_form_data, new_form_data, config)
+
+ for key, value in existing_form_data.items():
+ if key in {"where", "having", "filters"}:
+ # merge_update_form_data already resolved these legacy predicates
+ # into adhoc_filters; restoring them would undo replacement/clear.
+ continue
+ if key == MCP_DASHBOARD_TIME_FILTER_SUBJECT:
+ # merge_update_form_data owns this provenance marker. Its absence
+ # may be an intentional subject clear and must not be undone by the
+ # generic preservation layer.
+ continue
+ if key not in new_form_data:
+ new_form_data[key] = value
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet controls without reinterpreting inherited query roles.
+
+ Saved Explore state may contain SQL dimensions and SQL WHERE/SIMPLE HAVING
+ filters beyond the typed authoring surface. Omitted roles are validated by
+ the native query contract and compilation, not as newly authored physical
+ columns or SIMPLE WHERE filters. Only this validation copy excludes them
+ and native query metadata; compiled and persisted form data stays intact.
+ Explicit replacements, including ``[]``, retain strict typed validation.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ for native_query_key in ("url_params", "extra_form_data", "extra_filters"):
+ validation_data.pop(native_query_key, None)
+ if update_config is None or isinstance(update_config, BulletChartConfig):
+ if update_config is None or update_config.dimensions is None:
+ validation_data.pop("groupby", None)
+ if update_config is None or "filters" not in update_config.model_fields_set:
Review Comment:
On a fresh `update_chart_preview` (no `form_data_key`) with `temporal_column: null` and any `filters` value, the mapper writes `_mcp_dashboard_time_filter_subject: None`, there is no previous state merge to drop it, and this branch keeps the marker once `filters` is set. `BulletChartConfig` then raises "must be a physical column name" and the call returns `invalid_merged_bullet_state` instead of a preview. Should the validation copy drop a null marker regardless of `filters`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4245844315
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1537,6 +1537,71 @@ including an explicit clear.
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
Review Comment:
Placing this H2 after `## Next Steps` makes the four existing H3 sections below it ("Reporting query dates", "Overriding a saved chart's time grain", "Exploring large dashboard layouts", "Reading applied dashboard filters") render as subsections of "Bullet chart compatibility", so the table of contents and anchors nest them under Bullet. Could the Bullet section move up next to the other chart-plugin sections so those H3s stay where they were?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4221520152
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2412,101 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ escape_separator(label)
Review Comment:
A grouped XY chart with a comma in a metric label (say `metrics: ["Revenue, net"]`, `groupby: ["region"]`) won't get its category series. The result keys were already unescaped to `Revenue, net, EU` by the query context processor, but this prefix is built from the escaped `Revenue\, net`, so none of the fields match, the adapter returns `None`, and the generic preview plots only the first numeric series with no per-region coloring while still reporting success. Should the prefix match use the unescaped label (and escape only when building the Vega field path)?
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1408,6 +1421,129 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Inherited native SQL dimensions remain query expressions when updating metrics
+or `order_by`; reference their output labels to sort by them. Caller-supplied
+`dimensions` must still be physical columns, not SQL expressions.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling. Bullet metric strings use
+JavaScript numeric spellings: underscore separators and non-ASCII digits are
+rejected rather than interpreted as numbers. Bounded array-valued dimensions
+retain their raw values in data reads and exports; preview category labels use
+JavaScript string conversion (for example, `[1, 2]` displays as `1,2`). Native
+SQL metrics without a label use their SQL expression as the output label.
+
+Presentation-only updates preserve saved predicates, including legacy top-level
+`where`, `having`, and `filters`, and normalize the native `order_by_cols` alias.
+When both `orderby` and `order_by_cols` are saved, their sort entries are
+concatenated in form-data key order, including when `orderby` is empty.
+Explicit `filters: []` and `order_by: []` clear those inherited controls.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+Labels that become empty after sanitization retain their slots so later labels
+remain aligned with their corresponding values.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates. Omitted label controls preserve the saved state only when their
+corresponding value controls (`ranges`, `markers`, or `marker_lines`) are also
+omitted. Replacing a value control without its labels clears the saved labels;
+resend the labels to retain annotations with the replacement values.
+
+Saved native `ranges`, `markers`, and `marker_lines` controls ignore empty,
+non-numeric, and NaN tokens, matching Explore. If no numeric range remains, the
+preview uses the default band up to 110% of the largest measure. Unrelated
+updates preserve these saved controls. Newly authored typed lists must contain
+finite numbers; infinite native values and oversized controls remain errors.
+
+On same-dataset chart updates, `filters: []` clears both user filters and the
+generated dashboard-time binding. `temporal_column: null` clears only that
+generated binding, preserving user filters. Omitting those controls preserves
+the saved binding. Bullet Vega previews keep separate indexed rows even when
+dimension display labels are identical, and support both `SMART_NUMBER` and
+`SMART_NUMBER_SIGNED` number formats. Bullet preview numeric format
+precision is limited to 20 digits before formatting; raw data reads do not
+validate presentation formats.
+
+Native Bullet temporal filters retain the active filter's subject and range as a
+pair; `No filter` placeholders do not supply the subject of another active range.
+Typed configs support one such pair. Multiple active native temporal filters, or
+conflicts with explicit `temporal_column`/`time_range`, are rejected rather than
+silently dropping or moving a predicate.
+
+### Query result limits
+
+MCP chart tools accept up to **50,000 rows per query** and **100,000 total rows**
+across all queries in one result. The aggregate counts each returned data row,
+including repeated rows in different queries; it does not use `rowcount` or
+`total_rows` metadata. The row-shaped `indexnames`
+array emitted by Chart Data uses the same 50,000-entry limit, rather than the
+4,096-entry limit for other metadata arrays. Index entries count toward the
+shared row-data work budget and retain the 1 MiB aggregate metadata byte limit.
+Multi-dimension pivot index tuples are serialized as JSON arrays. Nested
+containers within an index entry retain the standard container limits.
+
+The shared query-result validator also enforces **non-configurable hard limits**:
+
+- **2,500,000 values/containers across all queries**. Each row object, cell
+ scalar (including null), and nested list, tuple, array, or object contributes
+ to this shared work budget; nested elements count individually and repeated
+ occurrences count again. Object keys count toward byte limits, not this value
+ count. Row-shaped `indexnames` and their entries also consume this budget.
+ For example, 50,000 rows with 50 scalar columns require 2,550,000 values
+ (50,000 row objects plus 2,500,000 cells) and exceed the budget even if their
+ encoded size is below 16 MiB.
+- **64 KiB (65,536 UTF-8 bytes) per row-cell string value**. A single
+ 65,537-byte cell fails even in a one-row result. Binary cells must also fit
+ the cell limit before and after conversion to UTF-8 or a `base64:` string.
+- **16 MiB (16,777,216 bytes) of aggregate JSON-encoded result data and
+ metadata** across all queries, including escaped strings, keys, and container
+ syntax. Metadata strings (including SQL text) are not subject to the row-cell
+ string cap; all metadata shares a separate 1 MiB aggregate allowance.
+
+These limits apply before rendering or exporting in `get_chart_data`,
+`get_chart_preview`, chart generation/update compile checks, `query_dataset`,
+and semantic-layer `get_table`. They include inline/table responses and MCP
+CSV, Excel, and Parquet export paths; exports do not bypass source validation.
Review Comment:
This says the limits cover MCP Parquet export paths (`UPDATING.md` line 41 repeats it), but no MCP tool exports Parquet: `GetChartDataRequest.format` only accepts `json`, `csv`, and `excel`, and `query_dataset`/`get_table` have no export option. An operator or client following this would try `format="parquet"` and get a request-validation error. Can both mentions be limited to CSV/Excel?
##########
superset/mcp_service/dataset/tool/query_dataset.py:
##########
@@ -390,23 +430,25 @@ async def query_dataset( # noqa: C901
% (len(data), len(raw_columns), query_duration_ms)
)
- return QueryDatasetResponse(
- from_dttm=query_result.get("from_dttm"),
- to_dttm=query_result.get("to_dttm"),
- dataset_id=dataset.id,
- dataset_name=dataset_name,
- columns=columns_meta,
- data=data,
- row_count=len(data),
- total_rows=query_result.get("rowcount"),
- summary=summary,
- performance=PerformanceMetadata(
- query_duration_ms=query_duration_ms,
- cache_status=cache_label,
- ),
- cache_status=cache_status,
- applied_filters=effective_filters,
- warnings=warnings,
+ return _bounded_response(
Review Comment:
Nothing exercises this `_bounded_response` guard through the tool. 255 rows with one 64 KiB `category` cell each pass source-result validation, but `columns[].sample_values` repeats those cells and pushes the full response past 16 MiB. The nearby test in `test_query_dataset.py` (around line 407) only covers an oversized cell rejected before profiling, and the shared preflight tests don't call this tool, so removing the wrapper would leave every test green while oversized success responses go out. Could a registered-tool test mock only query execution and assert a `MalformedQueryResult` error for that payload?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1738,564 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ # Native extractQueryFields excludes empty-string column references.
+ table_columns = [
+ column
+ for column in _as_list(
+ form_data.get("all_columns") if raw_mode else form_data.get("groupby")
+ )
+ if column != ""
+ ]
+ table_metrics = [] if raw_mode else _as_list(form_data.get("metrics"))
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ query_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ query_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ # Native comparisons use ordinary metrics, before percentage-only metrics
+ # are added to the selected query and contribution operator.
+ has_time_comparison = _time_comparison(form_data, table_metrics)
+ offsets = _table_time_offsets(form_data, {**query, "metrics": table_metrics})
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if has_time_comparison:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if has_time_comparison and offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ # ``query["row_limit"]`` is the normalized caller limit (explicit request
+ # limit or the saved row_limit, which may be stored as a string); page
+ # sizing narrows it but never replaces it.
+ configured_limit = _positive_int(query.get("row_limit"))
+ if form_data.get("server_pagination"):
+ if page_size := _positive_int(form_data.get("server_page_length")):
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if query_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ query_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
+ if metric is None:
+ plural_metrics = _as_list(form_data.get("metrics"))
+ metric = plural_metrics[0] if plural_metrics else None
+ columns = _resolve_big_number_query_columns(form_data) if trendline else []
+ query = build_single_query_dict(
+ form_data,
+ columns,
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if trendline:
+ # Big Number has no series dimension. Its frontend pivot receives
+ # the common base QueryObject (whose columns are empty), not the
+ # final QueryObject after the explicit x-axis is added. Preserve
+ # that distinction instead of falling back to the final columns.
+ query["series_columns"] = []
+ if not form_data.get("x_axis"):
+ query["is_timeseries"] = True
+ query["post_processing"] = _timeseries_post_processing(form_data, query)
+ if form_data.get("aggregation") == "raw":
+ return [
+ query,
+ {
+ **query,
+ "columns": [],
+ "is_timeseries": False,
+ "post_processing": [],
+ },
+ ]
+ return [query]
+
+
+def build_waterfall_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Waterfall buildQuery with raw-axis ordering."""
+ metrics, groupby = resolve_metrics_and_groupby(form_data)
+ # normalizeTimeColumn runs after Waterfall's buildQuery callback. It
+ # wraps only the final x-axis column; orderby deliberately retains the
+ # raw control value produced inside the callback.
+ raw_axis = form_data.get("x_axis") or form_data.get("granularity_sqla")
+ query_axis = (
+ _normalized_x_axis_query_field(form_data)
+ if form_data.get("x_axis")
+ else raw_axis
+ )
+ waterfall_columns = ([query_axis] if query_axis else []) + groupby
+ raw_ordering_columns = ([raw_axis] if raw_axis else []) + groupby
+ query = build_single_query_dict(
+ form_data,
+ waterfall_columns,
+ metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=None,
+ )
+ query["orderby"] = [[column, True] for column in raw_ordering_columns]
+ if form_data.get("x_axis"):
+ query.pop("is_timeseries", None)
+ return [query]
+
+
+def build_mixed_timeseries_query_dicts( # noqa: C901
Review Comment:
This new builder replaces `build_mixed_timeseries_secondary` (line 1695) and `with_x_axis_column` (line 2398), which no longer have any caller now that the Mixed Timeseries plugin uses it. The old builder lacks the shared-filter reconciliation and post-processing of the active one, so a later fix for secondary-layer filtering or ordering applied to the old one would change no MCP output. Can these two helpers be removed?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -760,7 +1639,56 @@ def build_single_query_dict(
order_desc if order_desc is not None else form_data.get("order_desc", True)
)
qd["orderby"] = [(sort_metric, not descending)]
+ if orderby:
+ qd["orderby"] = orderby
+ for key in (
+ "annotation_layers",
+ "row_offset",
+ "series_columns",
+ "group_others_when_limit_reached",
+ "is_timeseries",
+ "time_offsets",
+ "time_compare_full_range",
+ ):
+ if key in form_data and form_data[key] is not None:
+ qd[key] = form_data[key]
+
+ # ``buildQueryObject`` keeps the modern series-limit controls, falls back
+ # to their legacy Timeseries names, and defaults the limit to zero. A
+ # malformed modern metric does not mask a valid legacy metric.
+ series_limit = form_data.get("series_limit")
+ if series_limit is None:
+ series_limit = form_data.get("limit")
+ if series_limit is not None:
+ qd["series_limit"] = series_limit
+ series_limit_metric = form_data.get("series_limit_metric")
+ if not _is_query_form_metric(series_limit_metric):
+ series_limit_metric = form_data.get("timeseries_limit_metric")
+ if series_limit_metric is not None:
+ qd["series_limit_metric"] = series_limit_metric
+ if apply_chart_fields is not None:
+ apply_chart_fields(qd, effective_row_limit)
apply_form_data_filters_to_query(qd, form_data)
+ # Mirror the common ``buildQueryObject``/``extractExtras`` translation used
+ # by native frontend plugins. ``granularity_sqla`` is a form-data control,
+ # while QueryObject calls the field ``granularity``; the SQL time grain is
+ # carried inside ``extras`` rather than as a top-level query field.
+ if include_common_temporal:
+ granularity = form_data.get("granularity") or form_data.get("granularity_sqla")
+ if granularity:
+ qd["granularity"] = granularity
+ if time_grain := form_data.get("time_grain_sqla"):
+ qd["extras"] = {
+ **(qd.get("extras") or {}),
+ "time_grain_sqla": time_grain,
Review Comment:
For a semantic-view Table in aggregate mode grouped by a non-temporal column (`temporal_columns_lookup: {"country": false}`) with a cached `time_grain_sqla: "P1D"`, this copies the grain into `extras`. Explore's Table `buildQuery` drops it through `omitDormantGrain`, but this reconstruction doesn't, so `get_chart_data` fails in the semantic layer with "A time column must be specified when a time grain is provided." for a chart that renders in Explore. Should the Table path apply the same dormant-grain removal for semantic-view datasources?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4224609246
##########
superset/mcp_service/dataset/tool/query_dataset.py:
##########
@@ -390,23 +430,25 @@ async def query_dataset( # noqa: C901
% (len(data), len(raw_columns), query_duration_ms)
)
- return QueryDatasetResponse(
- from_dttm=query_result.get("from_dttm"),
- to_dttm=query_result.get("to_dttm"),
- dataset_id=dataset.id,
- dataset_name=dataset_name,
- columns=columns_meta,
- data=data,
- row_count=len(data),
- total_rows=query_result.get("rowcount"),
- summary=summary,
- performance=PerformanceMetadata(
- query_duration_ms=query_duration_ms,
- cache_status=cache_label,
- ),
- cache_status=cache_status,
- applied_filters=effective_filters,
- warnings=warnings,
+ return _bounded_response(
Review Comment:
Agreed. Added in d6137b1fc76c09d3be930140aa58cf0d61cf3c2a: `test_query_dataset_rejects_oversized_complete_response` calls the registered `query_dataset` tool with only `ChartDataCommand` mocked, using 255 rows with a unique 64 KiB `category` cell each. It first asserts `query_result_data` accepts that source result, then asserts the tool returns `MalformedQueryResult` with no `data`. With `_bounded_response` disabled locally, this is the only test in `test_query_dataset.py` that fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4175282560
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1781,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. Inherited Bullet sorts are pruned after the merge against final exact output roles, including native adhoc metric sorts. Replacing SavedRevenue removes its obsolete sort, and removing Region keeps only the exact region sort. The update regressions no longer mock normalization or _validate_update_against_dataset: real merged-state and dataset validation run; only live SQL execution is substituted.
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,427 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ matches = [
+ item
+ for item in saved_metrics
+ if item == name or item.casefold() == name.casefold()
+ ]
+ if len(set(matches)) != 1:
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. The native HAVING branch prefers exact saved-metric names before case-folded fallback. Regressions validate both exact Revenue and revenue with both metrics present, using the real native reference validator.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2538,6 +2539,89 @@ def _metric_display_label(col: ColumnRef) -> str:
return col.label or col.name or ""
+def _require_unique_bullet_order_match(
+ matches: list[tuple[str, int | None]], target: str, role: str
+) -> tuple[str, int | None] | None:
+ """Return a unique sort-role match or reject ambiguity."""
+ if len(matches) == 1:
+ return matches[0]
+ if len(matches) > 1:
+ raise ValueError(f"ambiguous {role}: {target}")
+ return None
+
+
+def _bullet_metric_output_label(metric: ColumnRef) -> str:
+ """Return the field name emitted by Bullet's native metric shape."""
+ if metric.saved_metric:
+ # Saved metrics are serialized as a bare metric-name string; a
+ # ColumnRef display label does not change the query output alias.
+ return metric.name or ""
+ return _metric_display_label(metric)
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. Bullet output-role validation and create_metric_object share aggregate normalization: omitted aggregates emit SUM, and STDDEV/VAR emit STDDEV_SAMP/VAR_SAMP. The regression groups by Revenue while sorting the actual aggregate output.
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,459 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+ if not data:
+ # The strict model's zero-valued ungrouped row is a render-only
+ # frontend fallback, not source query data, so an empty query
+ # exposes no rows to get-data and exports.
+ return [], None
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the dimensions. Its float measure is
+ # render-only: exported rows keep the validated source metric so exact
+ # BIGINT/Decimal values are not rounded to binary64.
+ rows: list[Any] = []
+ for source, row in zip(data, model.rows, strict=False):
+ exposed = dict(row)
+ if model.metric_field in source:
+ exposed[model.metric_field] = _safe_enum_backing(
+ source[model.metric_field]
+ )
+ rows.append(exposed)
+ return rows, None
+
+ def ascii_preview(
+ self, data: list[Any], form_data: dict[str, Any], width: int
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def vega_lite_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> VegaLitePreview | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_bullet_vega_lite_preview,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_bullet_vega_lite_preview(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def resolve_update_config(
+ self,
+ config: Any,
+ existing_form_data: dict[str, Any],
+ *,
+ dataset_rebind: bool,
+ ) -> Any:
+ """Resolve sort targets against the saved hierarchy before mapping."""
+ if not isinstance(config, BulletChartConfig) or config.dimensions is not None:
+ return config
+ if not config.order_by:
+ return config
+ dimensions = (
+ existing_form_data.get("groupby") or []
+ if not dataset_rebind and existing_form_data.get("viz_type") == "bullet"
+ else []
+ )
+ return BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": dimensions}
+ )
+
+ def merge_update_form_data(
+ self,
+ existing_form_data: dict[str, Any],
+ new_form_data: dict[str, Any],
+ config: Any,
+ *,
+ dataset_rebind: bool,
+ ) -> dict[str, Any] | None:
+ """Merge filter provenance and preserve omitted native Bullet controls."""
+ if not isinstance(config, BulletChartConfig) or dataset_rebind:
+ return None
+ from superset.mcp_service.chart.chart_utils import (
+ merge_bullet_form_data,
+ merge_update_form_data,
+ )
+
+ merged = dict(new_form_data)
+ merge_update_form_data(existing_form_data, merged, config)
+ merge_bullet_form_data(existing_form_data, merged)
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. The Bullet same-dataset merge preserves omitted url_params. Save and preview regressions retain the saved EU template input across a presentation-only change; dataset rebinds still use their separate merge contract.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1091,350 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = float(token)
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ category_field = _unique_bullet_category_field(model.rows)
+ row_field = _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_row", (category_field,)
+ )
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field, row_field)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[row_field] = row_index
+ copied[category_field] = (
+ ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ if model.dimensions
+ else ""
+ )
+ if (
+ range_tooltip_field is not None
+ and containing_range_labels[row_index] is not None
+ ):
+ copied[range_tooltip_field] = containing_range_labels[row_index]
+ values.append(copied)
+
+ # Explore uses indexed rows even when their display labels are identical.
+ category_labels = [row[category_field] for row in values]
+ vega_format = {
+ "SMART_NUMBER": "~s",
+ "SMART_NUMBER_SIGNED": "+~s",
+ }.get(model.y_axis_format, model.y_axis_format)
+ y_encoding = {
+ "field": row_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ "sort": None,
+ "axis": {"labelExpr": f"{json.dumps(category_labels)}[datum.value]"},
+ }
+ tooltip = [
+ {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ },
+ {
+ "field": model.metric_field,
+ "type": "quantitative",
+ "format": vega_format,
+ },
+ ]
+ if range_tooltip_field is not None:
+ tooltip.append(
+ {"field": range_tooltip_field, "type": "nominal", "title": "Range"}
+ )
+ axis_min = min(
+ 0.0,
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ axis_max = max(
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ if axis_min == axis_max:
+ axis_max = axis_min + (abs(axis_min) or 1)
+
+ def label_at(labels: list[str], index: int, value: float, prefix: str) -> str:
+ if index < len(labels) and labels[index]:
+ return labels[index]
+ return (
+ ""
+ if prefix == "Range"
+ else _format_bullet_number(model.y_axis_format, value)
+ )
+
+ def legend_color(name: str) -> dict[str, Any]:
+ return {
+ "datum": name,
+ "type": "nominal",
+ "legend": {"title": None} if model.show_legend else None,
+ }
+
+ layers: list[dict[str, Any]] = []
+ range_entries = sorted(
+ [
+ (
+ threshold,
+ label_at(model.range_labels, index, threshold, "Range"),
+ )
+ for index, threshold in enumerate(model.ranges)
+ ],
+ key=lambda entry: entry[0],
+ reverse=True,
+ )
+ for index, (threshold, label) in enumerate(range_entries):
+ layers.append(
+ {
+ "mark": {
+ "type": "rect",
+ "opacity": max(0.08, 0.28 - index * 0.04),
+ },
+ "encoding": {
+ "x": {"datum": axis_min, "type": "quantitative"},
+ "x2": {"datum": threshold},
+ "y": y_encoding,
+ "color": legend_color(
+ (f"{label}: " if label else "")
+ + f"≤ {_format_bullet_number(model.y_axis_format, threshold)}"
+ ),
+ "tooltip": [
+ {"value": label, "title": "Range"},
+ {
+ "value": _format_bullet_number(
+ model.y_axis_format, threshold
+ ),
+ "title": "Threshold",
+ },
+ ],
+ },
+ }
+ )
+ if model.show_labels and label:
+ layers.append(
+ {
+ "mark": {"type": "text", "align": "right", "dx": -3},
+ "encoding": {
+ "x": {"datum": threshold, "type": "quantitative"},
+ "y": y_encoding,
+ "text": {"value": label},
+ },
+ }
+ )
+ layers.append(
+ {
+ "mark": {"type": "bar", "tooltip": True, "size": 16},
+ "encoding": {
+ "x": {
+ "field": model.metric_field,
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. Bullet Vega metric field references escape dots, brackets, and backslashes while row keys, titles, and metadata remain literal. Regressions cover both the measure and its tooltip.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,455 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
+ ):
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
+
+
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
+ x_axis = form_data.get("x_axis")
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
+ },
+ }
+ )
+ rolling_type = form_data.get("rolling_type")
+ rolling_columns = (
+ [*offset_map.values(), *offset_map.keys()] if comparison else metric_labels
+ )
+ if rolling_type == "cumsum":
+ chain.append(
+ {
+ "operation": "cum",
+ "options": {
+ "operator": "sum",
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ elif rolling_type in {"sum", "mean", "std"}:
+ chain.append(
+ {
+ "operation": "rolling",
+ "options": {
+ "rolling_type": rolling_type,
+ "window": int(form_data.get("rolling_periods") or 1),
+ "min_periods": int(form_data.get("min_periods") or 0),
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ comparison_type = form_data.get("comparison_type")
+ if comparison and comparison_type != "values":
+ chain.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": list(offset_map.values()),
+ "compare_columns": list(offset_map.keys()),
+ "compare_type": comparison_type,
+ "drop_original_columns": True,
+ },
+ }
+ )
+ if complete_timeseries_contract and form_data.get("contributionMode"):
+ chain.append(
+ {
+ "operation": "contribution",
+ "options": {
+ "orientation": form_data["contributionMode"],
+ "time_shifts": offsets if comparison else [],
+ },
+ }
+ )
+ if comparison:
+ rename: dict[str, str | None] = {}
+ for shifted, metric in offset_map.items():
+ offset = next(
+ (item for item in offsets if shifted.endswith(f"__{item}")), None
+ )
+ source = (
+ shifted
+ if comparison_type == "values"
+ else f"{comparison_type}__{metric}__{shifted}"
+ )
+ rename[source] = f"{metric}, {offset}" if len(metrics) > 1 else offset
+ if rename:
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {"columns": rename, "level": 0, "inplace": True},
+ }
+ )
+ elif (
+ x_label
+ and len(metrics) == 1
+ and (series_labels or len(offsets) > 1)
+ and form_data.get("truncate_metric") is not None
+ and form_data.get("truncate_metric")
+ ):
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {
+ "columns": {metric_labels[0]: None},
+ "level": 0,
+ "inplace": True,
+ },
+ }
+ )
+ if complete_timeseries_contract:
+ x_axis_sort = form_data.get("x_axis_sort")
+ x_axis_sort_asc = form_data.get("x_axis_sort_asc")
+ sortable_labels = [
+ label
+ for label in [
+ x_label,
+ *(
+ _metric_label(metric)
+ for metric in _as_list(form_data.get("metrics"))
+ ),
+ *(
+ _metric_label(metric)
+ for metric in _timeseries_extra_metrics(form_data)
+ ),
+ ]
+ if label
+ ]
if (
- not isinstance(entry, (list, tuple))
- or len(entry) != 2
- or not isinstance(entry[0], str)
- or not entry[0]
- or not isinstance(entry[1], bool)
+ x_axis_sort is not None
+ and x_axis_sort_asc is not None
+ and x_axis_sort in sortable_labels
+ and not _as_list(form_data.get("groupby"))
):
- raise ValueError(
- f"Gantt order_by_cols[{index}] must be [column, ascending_boolean]"
- )
- orderby.append([entry[0], entry[1]])
+ options: dict[str, Any] = {"ascending": x_axis_sort_asc}
+ if x_axis_sort == x_label:
+ options["is_sort_index"] = True
+ else:
+ options["by"] = x_axis_sort
+ chain.append({"operation": "sort", "options": options})
+ chain.append({"operation": "flatten"})
+
+ if complete_timeseries_contract and form_data.get("forecastEnabled") and x_label:
+ x_axis_grain = (
+ x_axis.get("timeGrain")
+ if isinstance(x_axis, dict)
+ and x_axis.get("expressionType") in {"SIMPLE", "SQL"}
+ else None
+ )
+ time_grain = (
+ x_axis_grain
+ or (query.get("extras") or {}).get("time_grain_sqla")
+ or form_data.get("time_grain_sqla")
+ or "P1D"
+ )
+ chain.append(
+ {
+ "operation": "prophet",
+ "options": {
+ "time_grain": time_grain,
+ "periods": int(form_data.get("forecastPeriods", 10)),
+ "confidence_interval": float(
+ form_data.get("forecastInterval", 0.8)
+ ),
+ "yearly_seasonality": form_data.get("forecastSeasonalityYearly"),
+ "weekly_seasonality": form_data.get("forecastSeasonalityWeekly"),
+ "daily_seasonality": form_data.get("forecastSeasonalityDaily"),
+ "index": x_label,
+ },
+ }
+ )
+ return [operator for operator in chain if operator is not None]
- columns: list[Any] = []
- seen: set[str] = set()
- for column in (
- start_time,
- end_time,
- category,
- *series_columns,
- *tooltip_columns,
- *(entry[0] for entry in orderby),
+
+def _mixed_layer_form_data(
+ form_data: dict[str, Any], *, secondary: bool
+) -> dict[str, Any]:
+ """Mirror MixedTimeseries remove/retainFormDataSuffix for one layer."""
+ if not secondary:
+ return {
+ key: value for key, value in form_data.items() if not key.endswith("_b")
+ }
+ layer = {key: value for key, value in form_data.items() if not key.endswith("_b")}
+ for isolated_key in (
+ "metrics",
+ "groupby",
+ "orderby",
+ "limit",
+ "series_limit",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ "order_desc",
+ "row_limit",
+ "truncate_metric",
+ "time_compare",
+ "comparison_type",
+ "resample_method",
+ "resample_rule",
+ "rolling_type",
+ "rolling_periods",
+ "min_periods",
+ "show_empty_columns",
):
- key = utils_json.dumps(column, sort_keys=True, default=str)
- if key not in seen:
- seen.add(key)
- columns.append(column)
- return columns, list(raw_tooltip_metrics), orderby, series_columns
+ if f"{isolated_key}_b" not in form_data:
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. Query B retains shared row_limit when row_limit_b is absent, and an explicit suffixed limit overrides it. The query-builder regression verifies both legs against the frontend retainFormDataSuffix contract.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4176537872
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,459 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ **(
+ {"isColumnReference": True}
+ if str(form_data.get("datasource", "")).endswith("__semantic_view")
+ else {}
+ ),
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
+ ):
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
+
+
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
+ x_axis = form_data.get("x_axis")
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
+ },
+ }
+ )
+ rolling_type = form_data.get("rolling_type")
+ rolling_columns = (
+ [*offset_map.values(), *offset_map.keys()] if comparison else metric_labels
+ )
+ if rolling_type == "cumsum":
+ chain.append(
+ {
+ "operation": "cum",
+ "options": {
+ "operator": "sum",
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ elif rolling_type in {"sum", "mean", "std"}:
+ chain.append(
+ {
+ "operation": "rolling",
+ "options": {
+ "rolling_type": rolling_type,
+ "window": int(form_data.get("rolling_periods") or 1),
+ "min_periods": int(form_data.get("min_periods") or 0),
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ comparison_type = form_data.get("comparison_type")
+ if comparison and comparison_type != "values":
+ chain.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": list(offset_map.values()),
+ "compare_columns": list(offset_map.keys()),
+ "compare_type": comparison_type,
+ "drop_original_columns": True,
+ },
+ }
+ )
+ if complete_timeseries_contract and form_data.get("contributionMode"):
+ chain.append(
+ {
+ "operation": "contribution",
+ "options": {
+ "orientation": form_data["contributionMode"],
+ "time_shifts": offsets if comparison else [],
+ },
+ }
+ )
+ if comparison:
+ rename: dict[str, str | None] = {}
+ for shifted, metric in offset_map.items():
+ offset = next(
+ (item for item in offsets if shifted.endswith(f"__{item}")), None
+ )
+ source = (
+ shifted
+ if comparison_type == "values"
+ else f"{comparison_type}__{metric}__{shifted}"
+ )
+ rename[source] = f"{metric}, {offset}" if len(metrics) > 1 else offset
+ if rename:
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {"columns": rename, "level": 0, "inplace": True},
+ }
+ )
+ elif (
+ x_label
+ and len(metrics) == 1
+ and (series_labels or len(offsets) > 1)
+ and form_data.get("truncate_metric") is not None
+ and form_data.get("truncate_metric")
+ ):
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {
+ "columns": {metric_labels[0]: None},
+ "level": 0,
+ "inplace": True,
+ },
+ }
+ )
+ if complete_timeseries_contract:
+ x_axis_sort = form_data.get("x_axis_sort")
+ x_axis_sort_asc = form_data.get("x_axis_sort_asc")
+ sortable_labels = [
+ label
+ for label in [
+ x_label,
+ *(
+ _metric_label(metric)
+ for metric in _as_list(form_data.get("metrics"))
+ ),
+ *(
+ _metric_label(metric)
+ for metric in _timeseries_extra_metrics(form_data)
+ ),
+ ]
+ if label
+ ]
if (
- not isinstance(entry, (list, tuple))
- or len(entry) != 2
- or not isinstance(entry[0], str)
- or not entry[0]
- or not isinstance(entry[1], bool)
+ x_axis_sort is not None
+ and x_axis_sort_asc is not None
+ and x_axis_sort in sortable_labels
+ and not _as_list(form_data.get("groupby"))
):
- raise ValueError(
- f"Gantt order_by_cols[{index}] must be [column, ascending_boolean]"
- )
- orderby.append([entry[0], entry[1]])
+ options: dict[str, Any] = {"ascending": x_axis_sort_asc}
+ if x_axis_sort == x_label:
+ options["is_sort_index"] = True
+ else:
+ options["by"] = x_axis_sort
+ chain.append({"operation": "sort", "options": options})
+ chain.append({"operation": "flatten"})
+
+ if complete_timeseries_contract and form_data.get("forecastEnabled") and x_label:
+ x_axis_grain = (
+ x_axis.get("timeGrain")
+ if isinstance(x_axis, dict)
+ and x_axis.get("expressionType") in {"SIMPLE", "SQL"}
+ else None
+ )
+ time_grain = (
+ x_axis_grain
+ or (query.get("extras") or {}).get("time_grain_sqla")
+ or form_data.get("time_grain_sqla")
+ or "P1D"
+ )
+ chain.append(
+ {
+ "operation": "prophet",
+ "options": {
+ "time_grain": time_grain,
+ "periods": int(form_data.get("forecastPeriods", 10)),
+ "confidence_interval": float(
+ form_data.get("forecastInterval", 0.8)
+ ),
+ "yearly_seasonality": form_data.get("forecastSeasonalityYearly"),
+ "weekly_seasonality": form_data.get("forecastSeasonalityWeekly"),
+ "daily_seasonality": form_data.get("forecastSeasonalityDaily"),
+ "index": x_label,
+ },
+ }
+ )
+ return [operator for operator in chain if operator is not None]
- columns: list[Any] = []
- seen: set[str] = set()
- for column in (
- start_time,
- end_time,
- category,
- *series_columns,
- *tooltip_columns,
- *(entry[0] for entry in orderby),
+
+def _mixed_layer_form_data(
+ form_data: dict[str, Any], *, secondary: bool
+) -> dict[str, Any]:
+ """Mirror MixedTimeseries remove/retainFormDataSuffix for one layer."""
+ if not secondary:
+ return {
+ key: value for key, value in form_data.items() if not key.endswith("_b")
+ }
+ layer = {key: value for key, value in form_data.items() if not key.endswith("_b")}
+ for isolated_key in (
+ "metrics",
+ "groupby",
+ "orderby",
+ "limit",
+ "series_limit",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ "order_desc",
+ "truncate_metric",
+ "time_compare",
+ "comparison_type",
+ "resample_method",
+ "resample_rule",
+ "rolling_type",
+ "rolling_periods",
+ "min_periods",
+ "show_empty_columns",
):
- key = utils_json.dumps(column, sort_keys=True, default=str)
- if key not in seen:
- seen.add(key)
- columns.append(column)
- return columns, list(raw_tooltip_metrics), orderby, series_columns
+ if f"{isolated_key}_b" not in form_data:
+ layer.pop(isolated_key, None)
Review Comment:
With shared `limit: 5`, `timeseries_limit_metric`, and `order_desc: false` but no corresponding `_b` overrides, this strips those controls from layer B even though Explore retains them, so the reconstructed secondary query ranks and returns a different set of series. Could unsuffixed controls remain unless a suffixed value overrides them?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,459 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ **(
+ {"isColumnReference": True}
+ if str(form_data.get("datasource", "")).endswith("__semantic_view")
+ else {}
+ ),
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
+ ):
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
+
+
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
+ x_axis = form_data.get("x_axis")
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
Review Comment:
A saved timeseries chart with `resample_fill_time_range: true` and observations only in the middle of the requested period loses its leading and trailing zero-filled days here, because the resample operator never receives `fill_time_range`. Could this preserve the flag that Explore sends so the server supplies the requested time boundaries?
##########
superset/mcp_service/semantic_layer/tool/get_table.py:
##########
@@ -366,23 +372,25 @@ def _build_response(
is_builtin: bool,
display_name: str,
query_result: dict[str, Any],
+ data: list[dict[str, Any]],
+ raw_columns: list[str],
+ coltypes: list[int | GenericDataType],
query_duration_ms: int,
warnings: list[str],
temporal_columns: set[str] | None = None,
valid_grains: dict[str, dict[str, str]] | None = None,
) -> GetTableResponse:
"""Format the query result into a GetTableResponse."""
- data = query_result.get("data", [])
- raw_columns = query_result.get("colnames", [])
cache_status = get_cache_status_from_result(
query_result, force_refresh=request.force_refresh
)
+ columns_meta = format_data_columns(data, raw_columns, coltypes)
Review Comment:
Every nonempty table response profiles all columns here and then discards that result when profiling again below with temporal overrides; a 5,000-row, 200-column result therefore repeats one million value inspections and their allocations. Could this first call be limited to the empty-result branch, leaving one profiling pass for nonempty data?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,532 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip()
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = float(Decimal(stripped))
Review Comment:
A saved/SQL metric returning `"1_000"` is accepted here as 1000, but Explore's `Number(...)` produces NaN, so compilation and the MCP preview succeed for a chart whose measure cannot render in the browser. Could metric-string validation reject Decimal-only spellings, including underscore separators and non-ASCII digits?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1897 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget, *, row_shaped: bool = False
+) -> str | None:
+ """Bound metadata, charging row-shaped indexes to the row-data work budget."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if row_shaped:
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "metadata contains too many total values"
+ else:
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = list.__len__(item)
+ # Chart Data emits one indexname per row. Only that outer array
+ # receives the row limit; containers within an index stay bounded.
+ max_items = (
+ MAX_QUERY_RESULT_ROWS
+ if row_shaped and depth == 0
+ else _MAX_ROW_CONTAINER_ITEMS
+ )
+ if width > max_items:
+ return "metadata contains an oversized array"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(width, mapping=False),
+ metadata=True,
+ ):
+ return reason
+ stack.extend(
+ (list.__getitem__(item, index), depth + 1, False)
+ for index in range(width)
+ )
+ continue
+
+ normalized, reason = _normalize_trusted_scalar(
+ item, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
Review Comment:
A successful pivot with two index dimensions produces `indexnames: [("US", "East")]` via `list(df.index)`, but this treats the tuple as an unsupported scalar and rejects the entire chart result, including JSON and CSV/Excel requests. Could bounded exact tuples be accepted as metadata containers as well as row-data containers?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5531961993
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4224297347
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,847 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
Review Comment:
Agreed. Fixed in 1a7455b21d4d6339beb650b6f093c4f7f55c5084: `map_bullet_config` now always writes `row_limit` (schema default 10000 when omitted), and the Bullet plugin update merge drops the mapper default when `row_limit` is not in `model_fields_set`, so `merge_bullet_form_data` restores the saved value. A saved chart with no stored limit falls back to the default, matching the shared merge path. Covered by `test_bullet_mapper_preserves_omission_and_honors_explicit_values` (omitted create gets 10000) and the new `test_bullet_update_merge_row_limit_omission_and_explicit_value` (omitted update keeps 15000, explicit 25 replaces it).
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2412,101 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ escape_separator(label)
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(label + FLAT_COLUMN_SEPARATOR)
Review Comment:
Confirmed: with `truncate_metric: true` and one metric the rename step leaves `{event_date, East, West}` and field discovery found nothing. Fixed in f19daaf51a2da8b6d9a3a36f0e3f9f1e3e3ab0e6: when no metric-prefixed field matches, there is a single metric, and `truncate_metric` is set, every non-x-axis column is folded as a category series. New `test_xy_preview_renders_truncated_metric_series` runs the real post-processing chain and asserts the fold covers `East` and `West`; it fails without the fix.
##########
superset/mcp_service/chart/compile.py:
##########
@@ -116,14 +119,19 @@ def _compile_chart(
row_limit=plugin.compile_row_limit(form_data) if plugin else 2,
force=False,
)
+ set_query_context_form_data(query_context, dataset_id, "table")
command = ChartDataCommand(query_context)
command.validate()
result = command.run()
warnings: List[str] = []
row_count = 0
- if query_failure := query_result_failure(result):
+ query_data, query_failure = query_result_data(
Review Comment:
Confirmed: master only checked failure statuses there, and the new value validation rejected `inf` before the Gauge normalizer could skip the dial. Fixed in c537e1add4: compile, the unsaved preview, and the saved ASCII/table/Vega-Lite previews pass `preserve_nonfinite_floats` from the owning plugin (true only for Gauge), the redundant `query_result_failure` re-checks in the saved previews are removed, and the saved Vega-Lite preview renders the normalized rows. Other plugins keep the strict check. New `test_compile_chart_skips_nonfinite_grouped_gauge_dial` and `test_unsaved_gauge_preview_skips_nonfinite_dial` (A=Infinity, B=42) fail without the fix and pass with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4228037162
##########
tests/unit_tests/mcp_service/chart/tool/test_update_chart_preview.py:
##########
@@ -1410,24 +1627,19 @@ async def test_validation_failure_skips_cache_write(
mock_create_form_data.assert_not_called()
@patch.object(update_chart_preview_module, "_find_dataset")
- @patch.object(update_chart_preview_module, "has_dataset_access", return_value=False)
- @patch("superset.daos.dataset.DatasetDAO.find_by_id")
@patch(
"superset.mcp_service.commands.create_form_data.MCPCreateFormDataCommand.run"
)
@pytest.mark.asyncio
async def test_dataset_access_denied_short_circuits(
self,
mock_create_form_data,
- mock_find_by_id,
- unused_access_mock,
mock_find_dataset,
mcp_server,
mock_auth,
):
- """has_dataset_access=False → DatasetNotAccessible, no cache write."""
- mock_find_dataset.return_value = _mock_dataset(id=3)
- mock_find_by_id.return_value = _mock_dataset(id=3)
+ """An inaccessible dataset short-circuits before mapping or cache writes."""
+ mock_find_dataset.return_value = None
Review Comment:
Fixed in 91019555303cd0571a604e2b1688826fcfc84f74. The merge brings back the tool's own `has_dataset_access` check in `update_chart_preview`. The test now keeps `_find_dataset` real, mocks `DatasetDAO.find_by_id` to return a dataset, and patches `has_dataset_access` to False. It asserts `DatasetNotAccessible`, that the access check was called with that dataset, and that neither `validate_and_compile` nor the cache write ran. If the access check is removed, the test fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4235896143
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -3383,7 +3414,620 @@ def _as_column_list(value: Any) -> list[Any]:
return [value]
-def analyze_chart_capabilities(viz_type: str | None, config: Any) -> ChartCapabilities:
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ # A missing saved viz_type is a boundary too, matching merge_chart_form_data.
+ if existing_form_data.get("viz_type") != new_form_data.get("viz_type"):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
Review Comment:
A range-only update on a chart whose temporal filter was saved from Explore (no MCP provenance marker) on a column other than the dataset's main time column won't reuse that filter. Here `existing_binding` is `None` (the marker lookup returns nothing, and the native-match fallback above only looks for the mapper's proposed subject), so the dataset-default binding is appended and the saved filter stays in place. E.g. a saved `ShipDate TEMPORAL_RANGE "Last year"` with main column `OrderDate`, updated with `time_range: "Last month"`, ends up with both filters, and `time_range: null` can't clear the `ShipDate` restriction. Should an unmarked, single native TEMPORAL_RANGE filter be adopted as the saved binding when `temporal_column` is omitted?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4234908254
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -3939,6 +4058,477 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ # Saved query roles are internal sort context, never typed authoring input.
+ # Keeping them out of dimensions preserves omission and native SQL shapes.
+ _inherited_groupby: list[str | dict[str, Any]] | None = PrivateAttr(default=None)
+
+ @property
+ def order_dimensions(self) -> Sequence[ColumnRef | str | dict[str, Any]]:
+ """Return authored dimensions or the inherited native sort hierarchy."""
+ if self.dimensions is None and self._inherited_groupby is not None:
+ return self._inherited_groupby
+ return self.dimensions or []
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
Review Comment:
`order_by` is declared with `AliasChoices("order_by", "orderby", "order_by_cols")`, and Pydantic keeps only the first alias that is present. Native Explore form data (and `_normalize_bullet_query_aliases`) concatenates `orderby` and `order_by_cols`, so a native config such as `metric: "sales"`, `groupby: ["region"]`, `orderby: []`, `order_by_cols: ['["sales", false]']`, `row_limit: 1` loses the descending-sales sort here and may return a lower-sales region. Should `adapt_native_form_data` merge these aliases into one list before field selection, the way the saved-form-data path does?
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -3939,6 +4058,477 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ # Saved query roles are internal sort context, never typed authoring input.
+ # Keeping them out of dimensions preserves omission and native SQL shapes.
+ _inherited_groupby: list[str | dict[str, Any]] | None = PrivateAttr(default=None)
+
+ @property
+ def order_dimensions(self) -> Sequence[ColumnRef | str | dict[str, Any]]:
+ """Return authored dimensions or the inherited native sort hierarchy."""
+ if self.dimensions is None and self._inherited_groupby is not None:
+ return self._inherited_groupby
+ return self.dimensions or []
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ temporal_pairs: list[tuple[str, str]] = []
+ inert_subjects: set[str] = set()
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ if not isinstance(comparator, str) or not comparator:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs a range"
+ )
+ if comparator.casefold() == "no filter":
+ inert_subjects.add(subject)
+ else:
+ temporal_pairs.append((subject, comparator))
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ if len(temporal_pairs) > 1:
+ raise ValueError(
+ "Multiple active native temporal filters cannot be represented by "
+ "a single temporal_column/time_range pair"
+ )
+ if temporal_pairs:
+ subject, comparator = temporal_pairs[0]
+ if data.get("temporal_column") not in (None, subject) or data.get(
+ "time_range"
+ ) not in (None, "No filter", comparator):
+ raise ValueError(
+ "Native temporal filter conflicts with temporal_column/time_range"
+ )
+ data["temporal_column"] = subject
+ data["time_range"] = comparator
+ elif len(inert_subjects) == 1:
+ data.setdefault("temporal_column", next(iter(inert_subjects)))
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
Review Comment:
This `viz_type == "bullet"` branch never runs for `generate_chart`/`update_chart` requests: `_normalize_chart_request_input` maps `viz_type` to `chart_type` and pops `viz_type` before this validator sees the config. So `{"config": {"viz_type": "bullet", "metric": "Revenue", "markers": "0x64"}}` skips `_bullet_numeric_control_tokens` and fails strict float validation, while the same dict validated directly against `BulletChartConfig` is accepted with marker 100. Should the request normalizer carry native provenance (as it does for `sunburst_v2`) so both entry points parse these controls the same way?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1809,7 +1815,1041 @@ def generate_xy_vega_lite_preview(
spec["transform"] = [
{"fold": series_columns, "as": [series_field, value_field]}
]
- encoding["y"] = {"field": value_field, "type": "quantitative"}
+ # Vega-Lite stacks bar and area marks colored by a nominal field
+ # by default; follow the chart's saved ``stack`` control instead.
+ encoding["y"] = {
+ "field": value_field,
+ "type": "quantitative",
+ "stack": _xy_vega_lite_stack(form_data.get("stack")),
+ }
encoding["color"] = {"field": series_field, "type": "nominal"}
return preview
+
+
+def _xy_vega_lite_stack(stack: Any) -> str | None:
+ """Map the ECharts timeseries ``stack`` control to a Vega-Lite stack."""
+ if not stack:
+ return None
+ return "normalize" if stack == "Expand" else "zero"
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+_MAX_BULLET_FIELDS = 256
+
+
+_MAX_BULLET_FIELD_BYTES = 1000
+
+
+_MAX_BULLET_TEXT_BYTES = 2000
+
+
+# ECMAScript WhiteSpace and LineTerminator characters used by trim/Number.
+_JAVASCRIPT_WHITESPACE = (
+ "\t\n\v\f\r \u00a0\u1680\u2000\u2001\u2002\u2003\u2004"
+ "\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000\ufeff"
+)
+
+
+_MAX_BULLET_TOKENS = 256
+
+
+_ENUM_SCALAR_TYPES = (str, int, float, bool, Decimal)
+
+
+class BulletOutputError(ValueError):
+ """A Bullet query result cannot be rendered without guessing its roles."""
+
+ def __init__(self, message: str, error_type: str = "MalformedBulletOutput") -> None:
+ super().__init__(message)
+ self.error_type = error_type
+
+
+@dataclass(frozen=True)
+class BulletRenderModel:
+ """Strict, frontend-aligned data and presentation roles for one preview."""
+
+ rows: list[dict[str, Any]]
+ metric_field: str
+ dimensions: list[str]
+ measures: list[float]
+ ranges: list[float]
+ range_labels: list[str]
+ markers: list[float]
+ marker_labels: list[str]
+ marker_lines: list[float]
+ marker_line_labels: list[str]
+ y_axis_format: str
+ show_labels: bool
+ show_legend: bool
+
+
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ # getColumnLabel: an adhoc column without a label is keyed by its SQL.
+ for key in ("label", "sqlExpression", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if type(value) in {list, tuple, dict}:
+ data, failure = first_query_data(
+ {"queries": [{"data": [{"value": value}]}]}, temporal_json_numbers=True
+ )
+ if failure is not None or data is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} "
+ "has an invalid or unbounded container value"
+ )
+ normalized = data[0]["value"]
+ reason = None
+ elif _is_chart_data_temporal_scalar(value := _safe_enum_backing(value)):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_scalar(value)
+ if (
+ reason is None
+ and normalized is None
+ and value is not None
+ and type(value) in {float, np.float16, np.float32, np.float64}
+ and math.isinf(float(value))
+ ):
+ # NaN keeps the shared null normalization, but an infinite source
+ # value has no faithful category text and is rejected.
+ reason = "contains a non-finite number"
+ if (
+ reason is None
+ and type(normalized) is str
+ and _bounded_utf8_length(normalized, _MAX_BULLET_TEXT_BYTES) is None
+ ):
+ reason = "exceeds the size limit"
+ if reason is not None:
+ if reason == "an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif not reason.startswith(("contains ", "exceeds ", "has ")):
+ reason = f"contains {reason}"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if value_type in {list, dict}:
+ text = _bullet_container_category_text(normalized, dimension, row_index)
+ elif normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_container_category_text(value: Any, dimension: str, row_index: int) -> str:
+ """Stringify validated containers like JavaScript with a bounded text budget."""
+ from superset.mcp_service.chart.query_result import _bounded_utf8_length
+
+ if type(value) is dict:
+ return "[object Object]"
+ parts: list[str] = []
+ size = 0
+ for index, item in enumerate(value):
+ if type(item) in {list, dict}:
+ text = _bullet_container_category_text(item, dimension, row_index)
+ else:
+ text = (
+ ""
+ if item is None
+ else _bullet_category_value(item, dimension, row_index)[1]
+ )
+ text_size = _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES)
+ size += (text_size if text_size is not None else _MAX_BULLET_TEXT_BYTES + 1) + (
+ index > 0
+ )
+ if size > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ parts.append(text)
+ return ",".join(parts)
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
Review Comment:
The schema lets `range_labels`/`marker_labels`/`marker_line_labels` carry up to 100 labels of 200 characters each, but this check caps the whole comma-joined string at `_MAX_BULLET_TEXT_BYTES` (2000). A config that passes validation, e.g. 21 range labels of 100 characters, gets joined into one control string of about 2,100 characters, so the render model raises `BulletOutputError` and `generate_chart`, saved-chart reads and exports fail with a malformed-output error even though the query returned valid numbers. Should the schema bound the total joined length, or this limit be raised to match the schema's per-label/count limits?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4240656660
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1809,7 +1822,1063 @@ def generate_xy_vega_lite_preview(
spec["transform"] = [
{"fold": series_columns, "as": [series_field, value_field]}
]
- encoding["y"] = {"field": value_field, "type": "quantitative"}
+ # Vega-Lite stacks bar and area marks colored by a nominal field
+ # by default; follow the chart's saved ``stack`` control instead.
+ encoding["y"] = {
+ "field": value_field,
+ "type": "quantitative",
+ "stack": _xy_vega_lite_stack(form_data.get("stack")),
+ }
encoding["color"] = {"field": series_field, "type": "nominal"}
+ if (
+ spec.get("mark") == "bar"
+ and encoding["y"]["stack"] is None
+ and "x" in encoding
+ ):
+ # Unstacked bars sit side by side like Explore's grouped bars.
+ # Offset channels need a discrete band x scale, so temporal
+ # values stay date-formatted through a millisecond time unit so
+ # sub-second timestamps keep distinct bands as in Explore.
+ x_encoding = encoding["x"]
+ if x_encoding.get("type") == "temporal":
+ x_encoding["timeUnit"] = (
+ "yearmonthdatehoursminutessecondsmilliseconds"
+ )
+ x_encoding["type"] = "ordinal"
+ encoding["xOffset"] = {"field": series_field, "type": "nominal"}
return preview
+
+
+def _xy_vega_lite_stack(stack: Any) -> str | None:
+ """Map the ECharts timeseries ``stack`` control to a Vega-Lite stack."""
+ if not stack:
+ return None
+ return "normalize" if stack == "Expand" else "zero"
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+_MAX_BULLET_FIELDS = 256
+
+
+_MAX_BULLET_FIELD_BYTES = 1000
+
+
+_MAX_BULLET_TEXT_BYTES = 2000
+
+
+# ECMAScript WhiteSpace and LineTerminator characters used by trim/Number.
+_JAVASCRIPT_WHITESPACE = (
+ "\t\n\v\f\r \u00a0\u1680\u2000\u2001\u2002\u2003\u2004"
+ "\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000\ufeff"
+)
+
+
+_MAX_BULLET_TOKENS = 256
+
+
+# A comma-joined presentation control holds up to _MAX_BULLET_TOKENS tokens of
+# _MAX_BULLET_TEXT_BYTES each, so the whole-control bound must cover that many
+# tokens plus separators rather than reuse the per-token bound.
+_MAX_BULLET_CONTROL_CHARS = _MAX_BULLET_TOKENS * (_MAX_BULLET_TEXT_BYTES + 1)
+
+
+_ENUM_SCALAR_TYPES = (str, int, float, bool, Decimal)
+
+
+class BulletOutputError(ValueError):
+ """A Bullet query result cannot be rendered without guessing its roles."""
+
+ def __init__(self, message: str, error_type: str = "MalformedBulletOutput") -> None:
+ super().__init__(message)
+ self.error_type = error_type
+
+
+@dataclass(frozen=True)
+class BulletRenderModel:
+ """Strict, frontend-aligned data and presentation roles for one preview."""
+
+ rows: list[dict[str, Any]]
+ metric_field: str
+ dimensions: list[str]
+ measures: list[float]
+ ranges: list[float]
+ range_labels: list[str]
+ markers: list[float]
+ marker_labels: list[str]
+ marker_lines: list[float]
+ marker_line_labels: list[str]
+ y_axis_format: str
+ show_labels: bool
+ show_legend: bool
+
+
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ # getColumnLabel: an adhoc column without a label is keyed by its SQL.
+ for key in ("label", "sqlExpression", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if type(value) in {list, tuple, dict}:
+ data, failure = first_query_data(
+ {"queries": [{"data": [{"value": value}]}]}, temporal_json_numbers=True
+ )
+ if failure is not None or data is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} "
+ "has an invalid or unbounded container value"
+ )
+ normalized = data[0]["value"]
+ reason = None
+ elif _is_chart_data_temporal_scalar(value := _safe_enum_backing(value)):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_scalar(value)
+ if (
+ reason is None
+ and normalized is None
+ and value is not None
+ and type(value) in {float, np.float16, np.float32, np.float64}
+ and math.isinf(float(value))
+ ):
+ # NaN keeps the shared null normalization, but an infinite source
+ # value has no faithful category text and is rejected.
+ reason = "contains a non-finite number"
+ if (
+ reason is None
+ and type(normalized) is str
+ and _bounded_utf8_length(normalized, _MAX_BULLET_TEXT_BYTES) is None
Review Comment:
This 2,000-byte cap applies to every dimension value, and `sanitize_data_rows` reuses the same render model for `get_chart_data`. A saved Bullet grouped by a text column with a 2,001-byte value passes the shared per-cell limit and renders in Explore, but a JSON/CSV/Excel fetch would return a malformed-output error instead of the rows. Should the raw-data path keep to the source-result limits and apply this bound only to preview labels?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -3489,7 +3521,655 @@ def _as_column_list(value: Any) -> list[Any]:
return [value]
-def analyze_chart_capabilities(viz_type: str | None, config: Any) -> ChartCapabilities:
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ # A missing saved viz_type is a boundary too, matching merge_chart_form_data.
+ if existing_form_data.get("viz_type") != new_form_data.get("viz_type"):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+ if (
+ existing_binding is None
+ and existing_subject is None
+ and range_explicit
+ and not subject_authoritative
+ and "filters" not in explicit_fields
+ ):
+ # A range-only update on an unmarked chart adopts its saved temporal
+ # filter even when Explore bound it to a column other than the mapper's
+ # proposed subject. Zero or several native ranges stay ambiguous, so the
+ # proposed binding is added instead of guessing which one to rewrite.
+ unmarked_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("expressionType") == "SIMPLE"
+ and filter_.get("clause") == "WHERE"
+ and isinstance(filter_.get("subject"), str)
+ ]
+ if len(unmarked_matches) == 1:
+ existing_binding = unmarked_matches[0]
+ existing_subject = existing_binding["subject"]
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
Review Comment:
When `filters` and `time_range` are sent together on a saved Bullet that has no temporal binding (no marker, no Explore-written range) and `temporal_column` is omitted, this branch sets `chosen_binding = existing_binding` (`None`), so the mapper's generated range filter is dropped. For example, `update_chart` with `filters: [{column: region, op: "=", value: North}]` and `time_range: "Last month"` would save only the region filter and the query would aggregate every month. Should this fall back to `incoming_binding` when `range_explicit` and there is no existing binding?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4236822642
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,568 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from copy import deepcopy
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ bullet_groupby_list,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ TablePreview,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Validate Bullet result rows without requiring a backend preview formatter."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data), validate_format=False)
+ except BulletOutputError as ex:
+ return ChartError(error=bounded_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+def _unsupported_table_preview() -> ChartError:
+ return ChartError(
+ error=(
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ ),
+ error_type="UnsupportedFormat",
+ )
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # A null ``data`` array is a malformed producer result, not an empty chart.
+ null_data_is_empty = False
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ validates_native_references = True
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ # A merged validation copy omits native dimensions; its native sorts
+ # belong to query-contract validation, not physical-column resolution.
+ orders = (
+ config_dict.get("order_by") or []
+ if config.dimensions is not None or config._inherited_groupby is not None
+ else []
+ )
+ for order in orders:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.order_dimensions, config.metric
+ )
+ if role == "dimension" and index is not None:
+ if config.dimensions is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized._inherited_groupby = deepcopy(config._inherited_groupby)
+ normalized.validate_roles_and_outputs()
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def build_query_dicts(
+ self,
+ form_data: dict[str, Any],
+ *,
+ viz_type: str,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+ ) -> list[dict[str, Any]] | None:
+ """Mirror ``Bullet/buildQuery``: one metric over the groupby hierarchy.
+
+ The frontend keeps only explicitly saved ordering; it never applies
+ the legacy top-N metric ordering of unadapted visualizations.
+ """
+ from superset.common.form_data_query_context import (
+ build_query_objects_from_form_data,
+ )
+
+ queries = build_query_objects_from_form_data(
+ form_data,
+ viz_type=viz_type,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ filters_prepared=True,
+ )
+ query = queries[0]
+ metric = form_data.get("metric")
+ query["columns"] = bullet_groupby_list(form_data.get("groupby"))
+ query["metrics"] = [metric] if metric else []
+ if not form_data.get("orderby") and not form_data.get("order_by_cols"):
+ query["orderby"] = []
+ return queries
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import first_query_data
+
+ rows, failure = first_query_data(
+ result, none_as_empty=False, temporal_json_numbers=True
+ )
+ if failure is not None:
+ return failure
+ assert rows is not None
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose validated rows independently of backend preview formatter support."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ model = resolve_bullet_render_model(
+ data, dict(form_data), validate_format=False
+ )
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+ if not data:
+ # The strict model's zero-valued ungrouped row is a render-only
+ # frontend fallback, not source query data, so an empty query
+ # exposes no rows to get-data and exports.
+ return [], None
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the dimensions. Its float measure is
+ # render-only: exported rows keep the validated source metric so exact
+ # BIGINT/Decimal values are not rounded to binary64.
+ rows: list[Any] = []
+ for source, row in zip(data, model.rows, strict=False):
+ exposed = dict(row)
+ if model.metric_field in source:
+ exposed[model.metric_field] = _safe_enum_backing(
+ source[model.metric_field]
+ )
+ rows.append(exposed)
+ return rows, None
+
+ def ascii_preview(
+ self,
+ data: list[Any],
+ form_data: dict[str, Any],
+ width: int,
+ height: int = 20,
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+
+ def table_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> TablePreview | ChartError | None:
+ return _unsupported_table_preview()
+
+ def unsupported_preview(self, preview_format: str) -> ChartError | None:
+ if preview_format == "table":
+ return _unsupported_table_preview()
+ return None
+
+ def vega_lite_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> VegaLitePreview | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_bullet_vega_lite_preview,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ return _generate_bullet_vega_lite_preview(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+
+ def resolve_update_config(
+ self,
+ config: Any,
+ existing_form_data: dict[str, Any],
+ *,
+ dataset_rebind: bool,
+ ) -> Any:
+ """Resolve sort targets against the saved hierarchy before mapping."""
+ if not isinstance(config, BulletChartConfig) or config.dimensions is not None:
+ return config
+ if not config.order_by:
+ return config
+ dimensions = (
+ bullet_groupby_list(existing_form_data.get("groupby"))
+ if not dataset_rebind and existing_form_data.get("viz_type") == "bullet"
+ else []
+ )
+ resolved = config.model_copy(deep=True)
+ resolved._inherited_groupby = deepcopy(dimensions)
+ return resolved.validate_roles_and_outputs()
+
+ def merge_update_form_data(
+ self,
+ existing_form_data: dict[str, Any],
+ new_form_data: dict[str, Any],
+ config: Any,
+ *,
+ dataset_rebind: bool,
+ ) -> dict[str, Any] | None:
+ """Merge filter provenance and preserve omitted native Bullet controls."""
+ if not isinstance(config, BulletChartConfig) or dataset_rebind:
+ return None
+ from superset.mcp_service.chart.chart_utils import (
+ _normalize_bullet_query_aliases,
+ merge_bullet_form_data,
+ merge_update_form_data,
+ )
+
+ saved_form_data = existing_form_data
+ if "orderby" in new_form_data:
+ # An explicit ordering replaces every saved ordering alias, so a
+ # stale or malformed saved alias is never parsed or inherited.
+ existing_form_data = {
+ key: value
+ for key, value in existing_form_data.items()
+ if key not in {"orderby", "order_by_cols"}
+ }
+ if existing_form_data.get("viz_type") == "bullet":
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ merged = dict(new_form_data)
+ if "row_limit" not in config.model_fields_set:
+ # The mapper always emits the schema default; an omitted limit
+ # keeps the saved value instead.
+ merged.pop("row_limit", None)
+ merge_update_form_data(existing_form_data, merged, config)
+ merge_bullet_form_data(existing_form_data, merged)
+ if "row_limit" not in merged and "row_limit" in new_form_data:
+ merged["row_limit"] = new_form_data["row_limit"]
+ # Like the shared overlay, unmodeled native presentation controls
+ # survive a same-viz update; query roles of other visualizations and
+ # legacy predicate aliases (already folded into adhoc filters) do not.
+ for key, value in saved_form_data.items():
+ if (
+ key not in merged
+ and key not in self.query_role_keys
+ and key not in {"where", "having", "filters"}
+ ):
+ merged[key] = value
Review Comment:
Confirmed and fixed in 7161e56ea506dd89fee5e81c16369a1969f86fc3. The same-viz restoration loop in `merge_update_form_data` now skips `_mcp_dashboard_time_filter_subject` whenever `adhoc_filters` is in the merged state, so the marker follows the merged filter sequence. Regression: `test_bullet_saved_update_filter_clear_drops_time_binding_marker` (immediate save and preview-first) asserts `filters: []` drops the marker and passes `validate_merged_bullet_form_data`; it failed before the fix.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1809,7 +1815,1047 @@ def generate_xy_vega_lite_preview(
spec["transform"] = [
{"fold": series_columns, "as": [series_field, value_field]}
]
- encoding["y"] = {"field": value_field, "type": "quantitative"}
+ # Vega-Lite stacks bar and area marks colored by a nominal field
+ # by default; follow the chart's saved ``stack`` control instead.
+ encoding["y"] = {
Review Comment:
Confirmed and fixed in 7161e56ea506dd89fee5e81c16369a1969f86fc3. When the folded mark is `bar` and `stack` resolves to None, the preview adds `xOffset` on the series field and makes x a discrete band (temporal x keeps date formatting via a full-precision `timeUnit`), so the bars sit side by side. Stacked bars and lines are unchanged. Regression: `test_wide_preview_unstacked_bars_are_offset_side_by_side`; the unstacked-bar case failed before the fix.
##########
superset/mcp_service/chart/compile.py:
##########
@@ -451,7 +456,474 @@ def _build_compile_error(message: str) -> ChartGenerationError:
)
-def validate_and_compile(
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ # Match QueryObject's guarded legacy saved-metric normalization.
+ if not ({"sqlExpression", "aggregate", "column"} & value.keys()):
+ label = value.get("label")
+ if isinstance(label, str) and label:
+ return "saved_metric", label
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", bounded_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
Review Comment:
Not reachable, agreed: Bullet is the only plugin with `validates_native_references`, and the dispatch baseline in `test_chart_plugin_contract.py` listed these `viz_type == 'mixed_timeseries'/'echarts_area'` branches. Opting other plugins in would change their update behavior outside this PR's scope, so 7161e56ea506dd89fee5e81c16369a1969f86fc3 removes the Timeseries/Mixed/Deck raw-control blocks and their baseline entries. The shared filter/QueryObject checks that Bullet uses are untouched; the chart suite passes.
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,568 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from copy import deepcopy
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ bullet_groupby_list,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ TablePreview,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Validate Bullet result rows without requiring a backend preview formatter."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data), validate_format=False)
+ except BulletOutputError as ex:
+ return ChartError(error=bounded_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+def _unsupported_table_preview() -> ChartError:
+ return ChartError(
+ error=(
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ ),
+ error_type="UnsupportedFormat",
+ )
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # A null ``data`` array is a malformed producer result, not an empty chart.
+ null_data_is_empty = False
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ validates_native_references = True
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ # A merged validation copy omits native dimensions; its native sorts
+ # belong to query-contract validation, not physical-column resolution.
+ orders = (
+ config_dict.get("order_by") or []
+ if config.dimensions is not None or config._inherited_groupby is not None
+ else []
+ )
+ for order in orders:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.order_dimensions, config.metric
+ )
+ if role == "dimension" and index is not None:
+ if config.dimensions is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized._inherited_groupby = deepcopy(config._inherited_groupby)
+ normalized.validate_roles_and_outputs()
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def build_query_dicts(
+ self,
+ form_data: dict[str, Any],
+ *,
+ viz_type: str,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+ ) -> list[dict[str, Any]] | None:
+ """Mirror ``Bullet/buildQuery``: one metric over the groupby hierarchy.
+
+ The frontend keeps only explicitly saved ordering; it never applies
+ the legacy top-N metric ordering of unadapted visualizations.
+ """
+ from superset.common.form_data_query_context import (
+ build_query_objects_from_form_data,
+ )
+
+ queries = build_query_objects_from_form_data(
+ form_data,
+ viz_type=viz_type,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ filters_prepared=True,
+ )
+ query = queries[0]
+ metric = form_data.get("metric")
+ query["columns"] = bullet_groupby_list(form_data.get("groupby"))
+ query["metrics"] = [metric] if metric else []
+ if not form_data.get("orderby") and not form_data.get("order_by_cols"):
+ query["orderby"] = []
+ return queries
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import first_query_data
+
+ rows, failure = first_query_data(
+ result, none_as_empty=False, temporal_json_numbers=True
+ )
+ if failure is not None:
+ return failure
+ assert rows is not None
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose validated rows independently of backend preview formatter support."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ model = resolve_bullet_render_model(
+ data, dict(form_data), validate_format=False
+ )
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+ if not data:
+ # The strict model's zero-valued ungrouped row is a render-only
+ # frontend fallback, not source query data, so an empty query
+ # exposes no rows to get-data and exports.
+ return [], None
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the dimensions. Its float measure is
+ # render-only: exported rows keep the validated source metric so exact
+ # BIGINT/Decimal values are not rounded to binary64.
+ rows: list[Any] = []
+ for source, row in zip(data, model.rows, strict=False):
Review Comment:
Confirmed and fixed in 7161e56ea506dd89fee5e81c16369a1969f86fc3. `sanitize_data_rows` no longer zips against the render-truncated model: for an ungrouped Bullet, every later source row is resolved through the same strict resolver and exported (the zip is now `strict=True`). Regression: `test_saved_ungrouped_bullet_get_data_keeps_every_source_row` drives `get_chart_data` with two ungrouped rows for JSON and CSV and asserts both survive; it returned one row before the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5792232815
@rusackas heads up — this grew after your approval, so flagging rather than sneaking it through.
Merging master picked up #44495, which routes MCP response values through `base_json_conv` → `float(obj)`. That turns `Decimal("0.10000000000000000001")` into `0.1` silently, which this PR has a test specifically forbidding.
Rather than drop the test, the fix went into the shared sanitizer (`superset/mcp_service/utils/serialization.py`): finite Decimals are preserved exactly, non-finite still serialize as `null`. Because pydantic has no exact JSON-*number* path for Decimal, preserved values go out as JSON strings — so Decimal columns change shape in `get_chart_data`, `get_dashboard_data`, `query_dataset`, `get_table` and `execute_sql`. Documented in UPDATING.md.
It's isolated in e7b105e5 if you'd prefer it split into its own PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5614164711
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5574516858
Finite follow-up complete at unchanged head `1e3150c8ae8d8cb5ed1ace38de6159cf7ac2558c`.
Compared the live snapshot with the [previous pass](https://github.com/apache/superset/pull/43770#issuecomment-5561052111) (2026-09-06 17:53:06 UTC): fetched all 20 reviews, 11 issue comments, 24 inline comments, and 12 review threads, with pagination exhausted. No new reviews or thread comments; all 12 threads remain resolved.
The only later issue-comment update is the Codecov report (74.53% patch coverage, 79.83% project coverage). Its exact-head checks conclude SUCCESS and contain no defect-specific annotations. Later lint/label checks also succeeded. The lint annotation concerns Node.js runtime deprecation in the unchanged shared PR-lint action, not this branch's changes. No confirmed material feedback warrants code or regression-test changes; no thread replies/resolutions or push were needed.
Exact-head verification:
- **13/13 required checks SUCCESS**, including [unit gate](https://github.com/apache/superset/actions/runs/34048205566/job/101531553509), [pre-commit](https://github.com/apache/superset/actions/runs/34048205559/job/101526965871), [Playwright gate](https://github.com/apache/superset/actions/runs/34048205583/job/101529555999), and [latest lint](https://github.com/apache/superset/actions/runs/34049982283/job/101531717457).
- Latest check runs: **82 success, 3 skipped, 2 neutral; no failed or pending checks**. Latest Netlify commit status also succeeded. Historical check runs were fetched separately; all report the pinned SHA.
- Worktree clean; `git diff --check` passed. `pre-commit run` exited successfully with all hooks skipped because there are no changed files. Tests were not rerun for this no-code pass; prior test evidence remains in the PR body.
- PR remains open, non-draft, and MERGEABLE; BLOCKED accompanies REVIEW_REQUIRED. No rebase, force-push, merge, or further review loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4129641723
##########
superset/mcp_service/chart/plugins/gauge.py:
##########
@@ -44,6 +49,12 @@ class GaugeChartPlugin(BaseChartPlugin):
native_viz_types: ClassVar[Mapping[str, str]] = {
"gauge_chart": "Gauge Chart",
}
+ requires_compile_check = True
+ strict_dataset_rebind = True
Review Comment:
Gauge implements its own `merge_update_form_data` (returning `merge_gauge_update_form_data`) but doesn't set `owns_update_merge = True` the way Gantt/Treemap do. Both update tools therefore still run the generic same-viz preservation pass afterward, which restores any key Gauge's own merge just removed -- because that pass only checks whether a key is present in the merged output, not whether it was intentionally cleared. So explicitly setting `time_range` or `granularity_sqla` to `null` to clear a saved Gauge threshold binding doesn't stick. Should `owns_update_merge = True` be set here?
##########
superset/mcp_service/chart/tool/update_chart.py:
##########
@@ -897,7 +975,6 @@ async def update_chart( # noqa: C901
if validation_config is not None and effective_norm_dataset_id is not None:
from superset.mcp_service.chart.validation.dataset_validator import (
DatasetValidator,
Review Comment:
This import list no longer includes `NORMALIZATION_EXCEPTIONS` -- and after checking, it now has no remaining callers anywhere in `superset/mcp_service/chart/` (the sibling `update_chart_preview.py` switched to inline exception tuples the same way). Its definition and comment in `dataset_validator.py` still describe it as shared by the validation pipeline and tools, which could mislead a future change to normalization exception handling. Could that dead constant and its comment be removed?
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -792,11 +833,21 @@ async def execute_chart_data( # noqa: C901
command.validate()
result = command.run()
- if form_data.get("viz_type") == "treemap_v2":
+ if _normalizes_data_results(form_data):
result = normalize_chart_query_result(result, form_data)
if isinstance(result, ChartError):
return result
- if query_failure := query_result_failure(result):
+ data_plugin = _data_plugin(chart_viz_type)
Review Comment:
`data_plugin` is resolved from the saved chart's `chart_viz_type` here, not from `effective_form_data`'s viz_type. When a `form_data_key` override supplies cached preview data for a different chart type (for example, previewing a type change to Bullet on a saved chart of another type), `allows_empty_result` and the row-shape flags below are taken from the old saved type's plugin instead of the type actually being queried, so a legitimately empty result can be rejected as `NoData` under the wrong contract. Should this resolve the plugin from the effective form data's `viz_type` instead?
##########
superset/mcp_service/chart/tool/generate_chart.py:
##########
@@ -85,6 +98,10 @@ async def generate_chart( # noqa: C901
- Set save_chart=True to permanently save the chart
- LLM clients MUST display returned chart URL to users
- Use numeric dataset ID or UUID (NOT schema.table_name format)
+ - MUST include chart_type in config (one of: 'xy', 'table', 'pie', 'bullet',
Review Comment:
This adds a second "MUST include chart_type" list that contradicts the existing one a few lines below: it says `'gauge_chart'` where the schema's discriminator is `'gauge'` (`GaugeChartConfig.chart_type: Literal["gauge"]`), and it's missing `'treemap_v2'`, `'bubble_v2'`, and `'gantt'`, which the existing list already has. A client following this list for Gauge would submit a value the schema rejects. Could `'bullet'` be added to the existing correct list below instead, and this duplicate list removed?
##########
superset/mcp_service/chart/tool/update_chart.py:
##########
@@ -72,6 +78,13 @@
logger = logging.getLogger(__name__)
+def _bounded_update_response(payload: object) -> GenerateChartResponse:
Review Comment:
`_bounded_update_response` never forwards a `persisted_chart_id` to `preflight_generate_chart_response`, unlike `generate_chart`'s equivalent wrapper, which passes `chart_id if request.save_chart else None`. So when an update's combined form data/previews exceed the response-size limit after the chart was already persisted, the response falls back to `chart: null, success: false` with nothing to indicate the update actually saved -- a client can't tell this apart from a failed update and may retry unnecessarily. Should this pass the chart's id through the same way `generate_chart` does?
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -581,6 +614,12 @@ async def execute_chart_data( # noqa: C901
# The query_context contains all the information needed to reproduce
# the chart's data exactly as shown in the visualization
query_context_json = None
+ form_data: dict[str, Any] = {}
+ if chart_params:
+ parsed_form_data = utils_json.loads(chart_params)
Review Comment:
This `utils_json.loads(chart_params)` isn't wrapped in a try/except, unlike the identical parse a few lines below it. A chart with malformed legacy `params` (e.g. `"{"`) now raises here instead of falling through to that guarded fallback, even when the chart has a valid saved `query_context` that would otherwise satisfy the request. Could this parse reuse the same `except (TypeError, ValueError)` guard as the block below?
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2656,6 +2740,426 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ data.setdefault("temporal_column", subject)
+ if isinstance(comparator, str) and comparator.casefold() != "no filter":
+ data.setdefault("time_range", comparator)
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
+ {"name": item} if isinstance(item, str) else item
+ for item in data[key]
+ ]
+ for key in ("order_by", "orderby", "order_by_cols"):
+ if key in data:
+ data[key] = cls._adapt_native_order_by(data[key])
+ cls._adapt_native_filters(data)
+ return data
+
+ @field_validator("ranges", "markers", "marker_lines", mode="before")
+ @classmethod
+ def tokenize_native_numeric_lists(cls, value: Any) -> Any:
+ """Parse numeric controls without creating values for empty tokens."""
+ if value is None:
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",") if token.strip()]
+ return value
+
+ @field_validator(
+ "range_labels", "marker_labels", "marker_line_labels", mode="before"
+ )
+ @classmethod
+ def tokenize_native_label_lists(cls, value: Any) -> Any:
+ """Parse label controls while preserving positional empty tokens."""
+ if value is None or value == "":
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",")]
+ return value
+
+ @field_validator("ranges", "markers", "marker_lines")
+ @classmethod
+ def reject_non_finite_values(cls, values: List[float]) -> List[float]:
+ if any(not math.isfinite(value) for value in values):
+ raise ValueError("Bullet thresholds and markers must be finite numbers")
+ return values
+
+ @field_validator("range_labels", "marker_labels", "marker_line_labels")
+ @classmethod
+ def validate_presentation_labels(cls, labels: List[str]) -> List[str]:
+ result: list[str] = []
+ for label in labels:
+ if "," in label:
+ raise ValueError(
+ "Bullet labels cannot contain commas because the frontend "
+ "comma-separated controls have no escaping"
+ )
+ if label == "":
+ result.append("")
+ continue
+ sanitized = sanitize_user_input(
+ label, "Bullet label", max_length=200, allow_empty=True
+ )
+ if sanitized is not None:
+ result.append(sanitized)
+ return result
+
+ @field_validator("time_range")
+ @classmethod
+ def sanitize_time_range(cls, value: str | None) -> str | None:
+ return sanitize_user_input(
+ value, "Time range", max_length=1000, allow_empty=True
+ )
+
+ @model_validator(mode="after")
+ def validate_roles_and_outputs(self) -> "BulletChartConfig": # noqa: C901
+ dimensions = self.dimensions or []
+ seen_names: set[str] = set()
+ for index, dimension in enumerate(dimensions):
+ _reject_sql_expression_on_dimension(dimension, f"dimensions[{index}]")
+ if dimension.saved_metric or dimension.aggregate:
+ raise ValueError(
+ f"dimensions[{index}] must be a physical dimension, not a metric"
+ )
+ name = dimension.name or ""
+ if name in seen_names:
+ raise ValueError(f"Duplicate Bullet dimension: {name!r}")
+ seen_names.add(name)
+
+ # ``map_bullet_config`` deliberately emits physical groupby names. The
+ # frontend therefore reads dimension results under those names even when
+ # a friendly ``ColumnRef.label`` was supplied. Only the metric is emitted
+ # with an output alias. Keep validation aligned with those actual result
+ # fields instead of treating dimension display labels as SQL aliases.
+ if (metric_output := _bullet_metric_output_label(self.metric)) in seen_names:
+ raise ValueError(
+ f"Bullet metric output label {metric_output!r} conflicts with a "
+ "dimension (its physical output name); provide a unique metric label"
+ )
+
+ resolved_order: list[tuple[str, int | None]] = []
+ for item in self.order_by:
Review Comment:
`order_by` is checked against `self.dimensions` here, before the saved dimension hierarchy is merged in by `update_chart`/`update_chart_preview`. Updating a saved Bullet chart's `order_by` while omitting `dimensions` -- the documented way to preserve the saved hierarchy -- fails with "order_by must reference a Bullet dimension or metric output" even though the referenced column exists in the chart's saved dimensions. Could this cross-check be deferred to the merge step, where the saved hierarchy is available?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5922915776
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5913188267
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5925791824
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4180181683
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,733 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
Review Comment:
A presentation-only Bullet update omits saved top-level `where`/`having`/`filters` and the supported `order_by_cols` alias from this preservation list, so it can persist an unfiltered chart or change which category survives a row limit. Could inherited native predicates and ordering aliases be normalized before the omission-aware merge, while retaining explicit clear semantics?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1119,357 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = value.split(",")
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str:
+ token = token.strip(_JAVASCRIPT_WHITESPACE)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = (
+ _javascript_numeric_string(token)
+ if type(token) is str
+ else float(token)
+ )
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ metric_reference = "".join(
+ "\\" + char if char in ".[]\\" else char for char in model.metric_field
+ )
+ category_field = _unique_bullet_category_field(model.rows)
+ row_field = _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_row", (category_field,)
+ )
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field, row_field)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[row_field] = row_index
+ copied[category_field] = (
+ ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ if model.dimensions
+ else ""
+ )
+ if (
+ range_tooltip_field is not None
+ and containing_range_labels[row_index] is not None
+ ):
+ copied[range_tooltip_field] = containing_range_labels[row_index]
+ values.append(copied)
+
+ # Explore uses indexed rows even when their display labels are identical.
+ category_labels = [row[category_field] for row in values]
+ vega_format = {
+ "SMART_NUMBER": "~s",
+ "SMART_NUMBER_SIGNED": "+~s",
+ }.get(model.y_axis_format, model.y_axis_format)
+ y_encoding = {
+ "field": row_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ "sort": None,
+ "axis": {"labelExpr": f"{json.dumps(category_labels)}[datum.value]"},
+ }
+ tooltip = [
+ {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ },
+ {
+ "field": metric_reference,
+ "type": "quantitative",
+ "title": model.metric_field,
+ "format": vega_format,
+ },
+ ]
+ if range_tooltip_field is not None:
+ tooltip.append(
+ {"field": range_tooltip_field, "type": "nominal", "title": "Range"}
+ )
+ axis_min = min(
+ 0.0,
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ axis_max = max(
+ [*model.measures, *model.ranges, *model.markers, *model.marker_lines]
+ )
+ if axis_min == axis_max:
+ axis_max = axis_min + (abs(axis_min) or 1)
+
+ def label_at(labels: list[str], index: int, value: float, prefix: str) -> str:
+ if index < len(labels) and labels[index]:
+ return labels[index]
+ return (
+ ""
+ if prefix == "Range"
+ else _format_bullet_number(model.y_axis_format, value)
+ )
+
+ def legend_color(name: str) -> dict[str, Any]:
+ return {
+ "datum": name,
+ "type": "nominal",
+ "legend": {"title": None} if model.show_legend else None,
+ }
+
+ layers: list[dict[str, Any]] = []
+ range_entries = sorted(
+ [
+ (
+ threshold,
+ label_at(model.range_labels, index, threshold, "Range"),
+ )
+ for index, threshold in enumerate(model.ranges)
+ ],
+ key=lambda entry: entry[0],
+ reverse=True,
+ )
+ for index, (threshold, label) in enumerate(range_entries):
+ layers.append(
+ {
+ "mark": {
+ "type": "rect",
+ "opacity": max(0.08, 0.28 - index * 0.04),
+ },
+ "encoding": {
+ "x": {"datum": axis_min, "type": "quantitative"},
+ "x2": {"datum": threshold},
+ "y": y_encoding,
+ "color": legend_color(
+ (f"{label}: " if label else "")
+ + f"≤ {_format_bullet_number(model.y_axis_format, threshold)}"
+ ),
+ "tooltip": [
Review Comment:
These range tooltip arrays contain constant `value` definitions, but Vega-Lite accepts field definitions in tooltip arrays and skips these entries, so range/marker hover details disappear and schema-validating consumers reject the preview. Could the range, marker, and marker-line layers use valid tooltip definitions?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1739,551 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
+ (form_data.get("all_columns") or [])
+ if raw_mode
+ else (form_data.get("groupby") or [])
+ )
+ table_metrics = [] if raw_mode else list(form_data.get("metrics") or [])
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ offsets = _table_time_offsets(form_data, query)
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if offsets:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ # ``query["row_limit"]`` is the normalized caller limit (explicit request
+ # limit or the saved row_limit, which may be stored as a string); page
+ # sizing narrows it but never replaces it.
+ configured_limit = _positive_int(query.get("row_limit"))
+ if form_data.get("server_pagination"):
+ if page_size := _positive_int(form_data.get("server_page_length")):
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if table_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ table_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
Review Comment:
Saved or migrated Big Number charts with only `metrics: ["count"]` now reconstruct an empty query and fail previews/data fallback, although `BigNumberChartPlugin.resolve_query_fields` still explicitly supports that saved shape. Could this builder retain its plural-metric fallback when `metric` is absent?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,554 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ from superset.utils.number_format import D3_FORMAT_RE
+
+ # Specifier length does not bound its requested output precision. Check the
+ # parsed precision before the formatter can allocate or round any value.
+ match = D3_FORMAT_RE.match(format_)
+ if match and match.group(8) is not None and int(match.group(8)) > 20:
+ raise BulletOutputError(
+ "Bullet number format precision must not exceed 20",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
Review Comment:
Formatting a valid `.20f` Bullet measure of `10000000000.0` raises `decimal.InvalidOperation` with the default Decimal precision, so this mandatory validation aborts previews and even raw data/export reads despite the query succeeding. Could accepted bounded formats use sufficient Decimal precision for the measure before this formatter is called?
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,450 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ # Match QueryObject's guarded legacy saved-metric normalization.
+ if not ({"sqlExpression", "aggregate", "column"} & value.keys()):
+ label = value.get("label")
+ if isinstance(label, str) and label:
+ return "saved_metric", label
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
+ viz_type = form_data.get("viz_type")
+ if strict_all_form_refs and (
+ viz_type == "mixed_timeseries"
+ or (
+ isinstance(viz_type, str)
+ and (
+ viz_type.startswith("echarts_timeseries") or viz_type == "echarts_area"
+ )
+ )
+ ):
+ if (raw_x_axis := form_data.get("x_axis")) is not None and (
+ error := column_error(raw_x_axis, "form-data x_axis column")
+ ):
+ return error
+ metric_fields = [
+ "metrics",
+ "size",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ ]
+ if viz_type == "mixed_timeseries":
+ metric_fields.extend(
+ [
+ "metrics_b",
+ "size_b",
+ "timeseries_limit_metric_b",
+ "series_limit_metric_b",
+ ]
+ )
+ for field_name in metric_fields:
+ raw_value = form_data.get(field_name)
+ values = raw_value if isinstance(raw_value, list) else [raw_value]
+ for value in values:
+ if value is not None and (
+ error := metric_error(value, f"form-data {field_name} metric")
+ ):
+ return error
+
+ layer_suffixes = ("", "_b") if viz_type == "mixed_timeseries" else ("",)
+ for suffix in layer_suffixes:
+ sort_field = f"x_axis_sort{suffix}"
+ if sort_field not in form_data or form_data.get(sort_field) is None:
+ continue
+ x_axis = form_data.get(f"x_axis{suffix}", form_data.get("x_axis"))
+ allowed_labels: set[str] = set()
+ if x_axis_label := _native_column_label(x_axis):
+ allowed_labels.add(x_axis_label)
+ raw_metrics = form_data.get(f"metrics{suffix}")
+ for metric in raw_metrics if isinstance(raw_metrics, list) else []:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ raw_limit_metric = form_data.get(f"timeseries_limit_metric{suffix}")
+ limit_metrics = (
+ raw_limit_metric
+ if isinstance(raw_limit_metric, list)
+ else [raw_limit_metric]
+ )
+ for metric in limit_metrics:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ sort_value = form_data[sort_field]
+ if not isinstance(sort_value, str) or sort_value not in allowed_labels:
+ return _native_validation_error(sort_field, repr(sort_value)[:200])
+
+ if (
+ strict_all_form_refs
+ and isinstance(viz_type, str)
+ and viz_type.startswith("deck_")
+ ):
+ # Deck layers store most query roles outside common columns/metrics.
+ # Validate every renderer-consumed raw control as well as the generated
+ # QueryObject so a dataset-only rebind cannot hide or discard a stale
+ # tooltip, cross-filter, spatial, path, or metric reference.
+ for spatial_field in ("spatial", "start_spatial", "end_spatial"):
+ spatial = form_data.get(spatial_field)
+ if spatial is None:
+ continue
+ if not isinstance(spatial, dict):
+ return _native_validation_error(
+ f"form-data {spatial_field}", repr(spatial)[:200]
+ )
+ spatial_type = spatial.get("type")
+ if not isinstance(spatial_type, str):
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ role_fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if role_fields is None:
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ for role_field in role_fields:
+ spatial_value = spatial.get(role_field)
+ if spatial_value is None:
+ return _native_validation_error(
+ f"form-data {spatial_field}.{role_field} column", "missing"
+ )
+ if error := column_error(
+ spatial_value, f"form-data {spatial_field}.{role_field} column"
+ ):
+ return error
+
+ for field_name in (
+ "line_column",
+ "geojson",
+ "dimension",
+ "cross_filter_column",
+ ):
+ column_value = form_data.get(field_name)
+ if column_value is not None and (
+ error := column_error(column_value, f"form-data {field_name} column")
+ ):
+ return error
+
+ tooltip_contents = form_data.get("tooltip_contents")
+ if tooltip_contents is not None and not isinstance(tooltip_contents, list):
+ return _native_validation_error(
+ "form-data tooltip_contents", repr(tooltip_contents)[:200]
+ )
+ for index, item in enumerate(tooltip_contents or []):
+ tooltip_value: Any = None
+ if isinstance(item, str):
+ tooltip_value = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ tooltip_value = item.get("column_name")
+ if tooltip_value is not None and (
+ error := column_error(
+ tooltip_value, f"form-data tooltip_contents[{index}] column"
+ )
+ ):
+ return error
+
+ metric_values: list[tuple[str, Any]] = []
+ if viz_type not in {"deck_geojson", "deck_polygon"}:
+ for field_name in ("metrics", "metric", "size"):
+ raw_deck_metrics = form_data.get(field_name)
+ deck_metrics = (
+ raw_deck_metrics
+ if isinstance(raw_deck_metrics, list)
+ else [raw_deck_metrics]
+ )
+ metric_values.extend(
+ (f"form-data {field_name} metric", deck_metric)
+ for deck_metric in deck_metrics
+ if deck_metric is not None
+ )
+ if viz_type == "deck_polygon" and form_data.get("metric") is not None:
+ metric_values.append(("form-data metric metric", form_data.get("metric")))
+ fixed_metric_fields = (
+ ("point_radius_fixed",)
+ if viz_type in {"deck_scatter", "deck_polygon"}
+ else ()
+ ) + (("line_width",) if viz_type == "deck_path" else ())
+ for field_name in fixed_metric_fields:
+ fixed_value = form_data.get(field_name)
+ deck_metric: Any = (
+ fixed_value
+ if (
+ isinstance(fixed_value, str)
+ and fixed_value
+ and viz_type != "deck_polygon"
+ )
+ else None
+ )
+ if isinstance(fixed_value, dict) and fixed_value.get("type") == "metric":
+ deck_metric = fixed_value.get("value")
+ if deck_metric is not None:
+ metric_values.append((f"form-data {field_name} metric", deck_metric))
+ if viz_type == "deck_path" and form_data.get("breakpoint_metric") is not None:
+ metric_values.append(
+ (
+ "form-data breakpoint_metric metric",
+ form_data.get("breakpoint_metric"),
+ )
+ )
+ for role, deck_metric in metric_values:
+ if error := metric_error(deck_metric, role):
+ return error
+
+ for filter_ in form_data.get("adhoc_filters") or []:
+ if not isinstance(filter_, dict) or filter_.get("expressionType") != "SIMPLE":
+ continue
+ if not strict_all_form_refs and _is_inert_adhoc_filter(filter_):
+ continue
+ subject = filter_.get("subject")
+ clause = str(filter_.get("clause") or "WHERE").upper()
+ if clause == "HAVING" and isinstance(subject, str):
+ metric_matches = (
+ [subject]
+ if subject in saved_metrics
+ else [
+ name
+ for name in saved_metrics
+ if name.casefold() == subject.casefold()
+ ]
+ )
+ if len(metric_matches) == 1:
+ continue
+ if subject is not None and (
+ error := column_error(subject, "form-data filter column")
+ ):
+ return error
+ if filter_.get("operator") == "TEMPORAL_RANGE" and isinstance(subject, str):
+ try:
+ temporal = resolve_dataset_column(subject, dataset_context)
+ except ValueError:
+ temporal = None
+ if temporal is not None and not temporal.get("is_temporal", False):
+ return _native_validation_error("temporal filter column", subject)
+
+ # temporal_columns_lookup describes the entire datasource, not selected
+ # roles. The physical form/query column checks validate selected references.
+
+ for query_index, query in enumerate(queries, 1):
+ metric_labels: set[str] = set()
+ for column in query.get("columns") or []:
+ if error := column_error(column, f"query {query_index} column"):
+ return error
+ for column in query.get("series_columns") or []:
+ if error := column_error(column, f"query {query_index} series column"):
+ return error
+ for column in query.get("groupby") or []:
+ if error := column_error(column, f"query {query_index} groupby column"):
+ return error
+ for level in query.get("grouping_sets") or []:
+ for column in level:
Review Comment:
A Pivot with a Custom SQL dimension such as `UPPER(region)` labeled `RegionUpper` and totals enabled is rejected on a compatible dataset-only rebind because its grouping-set label is checked as a physical column. Could grouping-set references resolve against the selected logical column labels, as the native query path does?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1739,551 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
Review Comment:
A saved aggregate Table with `groupby: "region"` now reconstructs columns as `["r", "e", "g", "i", "o", "n"]`, making previews and fallback data/SQL reads fail; scalar `metrics` is split the same way. Could these roles use the existing scalar-to-list normalization rather than `list(...)`?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,554 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
Review Comment:
A Bullet grouped by an array-valued column returns values such as `[1, 2]` that pass the bounded result walker but are rejected by this scalar-only dimension conversion, breaking saved data reads and exports as well as previews. Could bounded container dimensions retain their raw values and use the frontend-compatible category text (`"1,2"` for this array)?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -589,23 +591,518 @@ def resolve_big_number_columns(form_data: dict[str, Any]) -> list[Any]:
return [granularity] if isinstance(granularity, str) and granularity else []
-def resolve_gantt_query_fields( # noqa: C901
- form_data: dict[str, Any],
-) -> tuple[list[Any], list[Any], list[list[Any]], list[Any]]:
- """Mirror the ECharts Gantt ``buildQuery`` field extraction contract.
+def extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
+ """Return the x_axis column name from form_data, or None if not set."""
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis:
+ return x_axis
+ if isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ return col_name if isinstance(col_name, str) and col_name else None
+ return None
+
- Returns ``(columns, metrics, orderby, series_columns)``. Saved form data is
- user-editable, so malformed or oversized native ordering is rejected rather
- than silently dropped or passed into ``QueryContextFactory``.
+def _x_axis_query_field(form_data: dict[str, Any]) -> Any | None:
+ """Resolve a frontend x-axis value without losing SQL expressions."""
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis:
+ return x_axis
+ if isinstance(x_axis, dict):
+ if (
+ isinstance(x_axis.get("sqlExpression"), str)
+ and x_axis.get("sqlExpression")
+ and isinstance(x_axis.get("label"), str)
+ and x_axis.get("label")
+ and x_axis.get("expressionType") in (None, "SQL")
+ ):
+ return x_axis
+ column_name = x_axis.get("column_name") or x_axis.get("columnName")
+ if isinstance(column_name, str) and column_name:
+ return column_name
+ return None
+
+
+def _normalized_x_axis_query_field(form_data: dict[str, Any]) -> Any | None:
+ """Mirror ``buildQueryContext.normalizeTimeColumn`` for a set x-axis."""
+ x_axis = _x_axis_query_field(form_data)
+ if x_axis is None:
+ return None
+ time_grain = form_data.get("time_grain_sqla")
+ if isinstance(x_axis, str):
+ normalized = {
+ "columnType": "BASE_AXIS",
+ "sqlExpression": x_axis,
+ "label": x_axis,
+ "expressionType": "SQL",
+ "isColumnReference": True,
+ }
+ if time_grain is not None:
+ normalized["timeGrain"] = time_grain
+ return normalized
+ normalized = {"columnType": "BASE_AXIS", **x_axis}
+ # The original adhoc column's grain overrides the common control, matching
+ # the frontend spread order.
+ if "timeGrain" not in normalized and time_grain is not None:
+ normalized["timeGrain"] = time_grain
+ return normalized
+
+
+def _resolve_big_number_query_columns(form_data: dict[str, Any]) -> list[Any]:
+ """Resolve only Big Number's explicit x-axis query column.
+
+ The frontend keeps ``granularity_sqla`` out of ``columns`` and asks the
+ backend for a timeseries instead, which yields ``__timestamp``. An explicit
+ ``x_axis`` is different: the plugin retains that column in the final query.
"""
- from superset.utils import json as utils_json
+ if (x_axis := _normalized_x_axis_query_field(form_data)) is not None:
+ return [x_axis]
+ return []
+
+
+def _as_list(value: Any) -> list[Any]:
+ """Match the frontend's ``ensureIsArray`` for query controls."""
+ if value is None:
+ return []
+ return value if isinstance(value, list) else [value]
+
+
+def _column_label(column: Any) -> str | None:
+ """Return the frontend ``getColumnLabel`` value for a query column."""
+ if isinstance(column, str):
+ return column
+ if not isinstance(column, dict):
+ return None
+ return (
+ column.get("label") or column.get("sqlExpression") or column.get("column_name")
+ )
+
+
+def _metric_label(metric: Any) -> str | None:
+ """Return the frontend ``getMetricLabel`` value for a query metric."""
+ if isinstance(metric, str):
+ return metric
+ if not isinstance(metric, dict):
+ return None
+ if label := metric.get("label"):
+ return label
+ if metric.get("expressionType") == "SIMPLE":
+ column = metric.get("column") or {}
+ name = (
+ column.get("columnName") or column.get("column_name")
+ if isinstance(column, dict)
+ else None
+ )
+ if name and metric.get("aggregate"):
+ return f"{metric['aggregate']}({name})"
+ return metric.get("sqlExpression")
+
+
+def _is_query_form_metric(value: Any) -> bool:
+ """Mirror the frontend's ``isQueryFormMetric`` type guard."""
+ return isinstance(value, str) or (
+ isinstance(value, dict) and value.get("expressionType") in {"SIMPLE", "SQL"}
+ )
+
+
+def _timeseries_base_metrics(form_data: dict[str, Any]) -> list[Any]:
+ """Return metrics extracted by the common frontend query-field aliases."""
Review Comment:
A saved timeseries chart using the backward-compatible singular `metric: "revenue"` now reconstructs with no metrics, so its MCP preview and fallback data/SQL query lose the measure. Could this preserve the singular metric alias supported by both the previous resolver and frontend `extractQueryFields`?
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,465 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
Review Comment:
A saved Bullet SQL metric without an explicit label previews correctly using its SQL expression as the result key, but this adapter supplies `label: None`, so a compatible dataset-only update fails `ColumnRef` validation before saving or previewing. Could native SQL metrics retain the expression-as-label fallback here too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4183053647
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,770 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop saved sorts that no longer target a final exact query output."""
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs = {
+ name for name in new_form_data.get("groupby") or [] if isinstance(name, str)
+ }
+ metrics = new_form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ for metric in [new_form_data.get("metric"), *metrics]:
+ label = metric.get("label") if isinstance(metric, Mapping) else metric
+ if isinstance(label, str):
+ outputs.add(label)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ if isinstance(target, Mapping):
+ target = target.get("label") or target.get("metric_name")
+ if isinstance(target, str) and target not in outputs:
+ continue
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if isinstance(existing_viz_type, str) and existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+def _currency_form_value(value: CurrencyFormat | None) -> dict[str, str] | None:
+ """Return the native value for an explicitly supplied currency control."""
+ return value.to_form_data() if value is not None else None
+
+
+def _column_names(value: Sequence[ColumnRef] | None) -> list[str | None] | None:
+ """Return a native column-name list while retaining an explicit null."""
+ return [column.name for column in value] if value is not None else None
+
+
+def _table_sort_value(value: Sequence[str | SortByConfig] | None) -> list[str] | None:
+ """Return the native Table sort control for an explicit typed value."""
+ if value is None:
+ return None
+ return [
+ json.dumps(
+ [entry.column, entry.ascending]
+ if isinstance(entry, SortByConfig)
+ else [entry, False]
+ )
+ for entry in value
+ ]
+
+
+def _table_column_config_value(value: Any) -> dict[str, Any] | None:
+ """Return Table column config without losing an explicit null or empty map."""
+ if value is None:
+ return None
+ return {
+ label: column.model_dump(by_alias=True, exclude_unset=True)
+ for label, column in value.items()
+ }
+
+
+# Mappers intentionally omit optional controls so fresh charts use the frontend
+# defaults. During a same-viz update, however, an explicitly supplied false,
+# null, or empty value must block preservation of the saved native key. Keep the
+# typed-to-native relationship declarative so every update path shares it.
+_FormValueConverter = Callable[[Any], Any]
+_FormControlMap = dict[str, tuple[str, _FormValueConverter]]
+
+_COMMON_EXPLICIT_FORM_CONTROLS: _FormControlMap = {
+ "color_scheme": ("color_scheme", lambda value: value),
+ "currency_format": ("currency_format", _currency_form_value),
+ "show_value": ("show_value", lambda value: value),
+}
+
+_CHART_EXPLICIT_FORM_CONTROLS: dict[str, _FormControlMap] = {
+ "table": {
+ "sort_by": ("order_by_cols", _table_sort_value),
+ "column_config": ("column_config", _table_column_config_value),
+ },
+ "xy": {
+ "group_by": ("groupby", _column_names),
+ "series_limit": ("series_limit", lambda value: value),
+ "stacked": ("stack", lambda value: "Stack" if value else None),
+ "orientation": ("orientation", lambda value: value),
+ "legend_orientation": ("legendOrientation", lambda value: value),
+ "x_axis_time_format": ("x_axis_time_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "mixed_timeseries": {
+ "group_by": ("groupby", _column_names),
+ "group_by_secondary": ("groupby_b", _column_names),
+ "currency_format_secondary": (
+ "currency_format_secondary",
+ _currency_form_value,
+ ),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "waterfall": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "big_number": {
+ "subheader": ("subheader", lambda value: value),
+ "y_axis_format": ("y_axis_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "compare_lag": ("compare_lag", lambda value: value),
+ "time_format": ("time_format", lambda value: value),
+ "aggregation": ("aggregation", lambda value: value),
+ },
+ "handlebars": {
+ "style_template": ("styleTemplate", lambda value: value),
+ "columns": ("all_columns", _column_names),
+ "groupby": ("groupby", _column_names),
+ "metrics": ("metrics", _column_names),
+ },
+ "pivot_table": {
+ "date_format": ("date_format", lambda value: value),
+ },
+ "interactive_pivot": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "series_limit": ("series_limit", lambda value: value),
+ "date_format": ("date_format", lambda value: value),
+ "column_sort": ("colOrder", lambda value: value),
+ },
+}
+
+
+def _apply_explicit_form_controls( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply typed controls whose mapper omission represents a native clear."""
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ controls = {
+ **_COMMON_EXPLICIT_FORM_CONTROLS,
+ **_CHART_EXPLICIT_FORM_CONTROLS.get(config.chart_type, {}),
+ }
+ for field_name, (native_key, convert) in controls.items():
+ if field_name in explicit_fields:
+ converted = convert(getattr(config, field_name))
+ is_clear = (
+ converted is None
+ or converted is False
+ or converted == ""
+ or converted in ([], {})
+ )
+ if is_clear:
+ new_form_data[native_key] = converted
+
+ axis_controls = {
+ "xy": (
+ ("x_axis", "x_axis_title", "x_axis_format", None),
+ ("y_axis", "y_axis_title", "y_axis_format", "logAxis"),
+ ),
+ "mixed_timeseries": (
+ ("x_axis", "xAxisTitle", "x_axis_time_format", None),
+ ("y_axis", "yAxisTitle", "y_axis_format", "logAxis"),
+ (
+ "y_axis_secondary",
+ "yAxisTitleSecondary",
+ "y_axis_format_secondary",
+ "logAxisSecondary",
+ ),
+ ),
+ }
+ if config.chart_type in axis_controls:
+ for field_name, title_key, format_key, scale_key in axis_controls[
+ config.chart_type
+ ]:
+ if field_name not in explicit_fields:
+ continue
+ axis = getattr(config, field_name)
+ if axis is None:
+ new_form_data[title_key] = None
+ new_form_data[format_key] = None
+ if scale_key:
+ new_form_data[scale_key] = None
+ continue
+ axis_fields = set(axis.model_fields_set)
+ if "title" in axis_fields:
+ new_form_data[title_key] = axis.title
+ if "format" in axis_fields:
+ new_form_data[format_key] = axis.format
+ if scale_key and "scale" in axis_fields:
+ new_form_data[scale_key] = (
+ None if axis.scale is None else axis.scale == "log"
+ )
+
+ if config.chart_type == "xy" and "legend" in explicit_fields:
+ legend = config.legend
+ if legend is None:
+ new_form_data["show_legend"] = None
+ new_form_data["legendOrientation"] = None
+ else:
+ legend_fields = set(legend.model_fields_set)
+ if "show" in legend_fields:
+ new_form_data["show_legend"] = legend.show
+ if "position" in legend_fields:
+ new_form_data["legendOrientation"] = legend.position
+
+ if config.chart_type == "interactive_pivot":
+ if "temporal_column" in explicit_fields and config.temporal_column is None:
+ new_form_data["granularity_sqla"] = None
+ new_form_data["temporal_columns_lookup"] = None
+ if (
+ "series_limit_metric" in explicit_fields
+ and config.series_limit_metric is None
+ ):
+ new_form_data["series_limit_metric"] = None
+ if "comparison_period" in explicit_fields and config.comparison_period is None:
+ new_form_data["time_compare"] = None
+ if "comparison_type" in explicit_fields and config.comparison_type is None:
+ new_form_data["comparison_type"] = None
+
+ # A Waterfall axis replacement cannot inherit a bucket belonging to the old
+ # temporal subject. Grain omission preserves only while the axis is stable;
+ # explicit null is already handled by the declarative control map above.
+ if (
+ config.chart_type == "waterfall"
+ and existing_form_data.get("x_axis") != new_form_data.get("x_axis")
+ and "time_grain" not in explicit_fields
+ ):
+ new_form_data["time_grain_sqla"] = None
+
+
+def merge_same_viz_form_data(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Preserve saved controls that the typed mapper does not represent.
+
+ The typed MCP surface deliberately models a bounded subset of every Explore
+ control panel. For a replacement within the exact same native ``viz_type``,
+ keys absent from the mapper therefore represent omitted controls and retain
+ their saved values. Mapper output and the chart-specific merge helpers run
+ first and remain authoritative, including explicit empty, false, null, and
+ nested values.
+
+ No generic state crosses a visualization boundary. This prevents query-role
+ keys from the previous plugin (for example ``metric`` or ``groupby``) from
+ leaking into a different plugin whose role contract is unrelated.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if not isinstance(existing_viz_type, str) or existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+
+ _apply_explicit_form_controls(existing_form_data, new_form_data, config)
+
+ for key, value in existing_form_data.items():
+ if key == MCP_DASHBOARD_TIME_FILTER_SUBJECT:
+ # merge_update_form_data owns this provenance marker. Its absence
+ # may be an intentional subject clear and must not be undone by the
+ # generic preservation layer.
+ continue
+ if key not in new_form_data:
+ new_form_data[key] = value
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet state without reclassifying preserved filters.
+
+ The typed Bullet surface intentionally creates only SIMPLE WHERE filters,
+ while saved Explore state may legitimately contain SQL WHERE or SIMPLE
+ HAVING filters. When an update omitted ``filters``, those native objects
+ came from the saved state and are validated by the form-data/query layer;
+ removing them only from this schema-validation copy avoids pretending they
+ were newly supplied typed filters. Explicit filter replacements, including
+ ``[]``, still take the strict native-to-typed path.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ preserves_native_filters = update_config is None or (
+ isinstance(update_config, BulletChartConfig)
+ and "filters" not in update_config.model_fields_set
+ )
+ if preserves_native_filters:
+ validation_data.pop("adhoc_filters", None)
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ return BulletChartConfig.model_validate(validation_data)
Review Comment:
Fixed in `74cad83129d2fde30b08dac585523da0087198bb`: `url_params` is excluded only from typed validation and retained for compilation and persistence.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,464 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ temporal_pairs: list[tuple[str, str]] = []
+ inert_subjects: set[str] = set()
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ if not isinstance(comparator, str) or not comparator:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs a range"
+ )
+ if comparator.casefold() == "no filter":
+ inert_subjects.add(subject)
+ else:
+ temporal_pairs.append((subject, comparator))
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ if len(temporal_pairs) > 1:
+ raise ValueError(
+ "Multiple active native temporal filters cannot be represented by "
+ "a single temporal_column/time_range pair"
+ )
+ if temporal_pairs:
+ subject, comparator = temporal_pairs[0]
+ if data.get("temporal_column") not in (None, subject) or data.get(
+ "time_range"
+ ) not in (None, "No filter", comparator):
+ raise ValueError(
+ "Native temporal filter conflicts with temporal_column/time_range"
+ )
+ data["temporal_column"] = subject
+ data["time_range"] = comparator
+ elif len(inert_subjects) == 1:
+ data.setdefault("temporal_column", next(iter(inert_subjects)))
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ from superset.mcp_service.chart.preview_utils import (
+ _bullet_numeric_control_tokens,
+ )
+
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ # Saved Explore controls are lenient; newly authored typed lists
+ # still undergo the strict List[float] validation below.
+ for control in ("ranges", "markers", "marker_lines"):
+ if control in data:
+ data[control] = _bullet_numeric_control_tokens(
+ data[control], control
+ )
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
Review Comment:
Fixed in `74cad83129d2fde30b08dac585523da0087198bb`: omitted inherited Bullet dimensions use the native query contract, preserving SQL hierarchies and their sorts.
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,462 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
Review Comment:
Fixed in `74cad83129d2fde30b08dac585523da0087198bb`: raw row validation skips preview-formatter checks, so `DURATION` and `MEMORY_BINARY` work for JSON, CSV, and Excel.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,770 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop saved sorts that no longer target a final exact query output."""
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs = {
+ name for name in new_form_data.get("groupby") or [] if isinstance(name, str)
+ }
+ metrics = new_form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ for metric in [new_form_data.get("metric"), *metrics]:
+ label = metric.get("label") if isinstance(metric, Mapping) else metric
+ if isinstance(label, str):
+ outputs.add(label)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ if isinstance(target, Mapping):
+ target = target.get("label") or target.get("metric_name")
+ if isinstance(target, str) and target not in outputs:
+ continue
+ retained.append(entry)
Review Comment:
Fixed in `74cad83129d2fde30b08dac585523da0087198bb`: inherited expression-based metric sorts are rebound to the final metric object when the output label is unchanged.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1739,557 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ # Native extractQueryFields excludes empty-string column references.
+ table_columns = [
+ column
+ for column in _as_list(
+ form_data.get("all_columns") if raw_mode else form_data.get("groupby")
+ )
+ if column != ""
+ ]
+ table_metrics = [] if raw_mode else _as_list(form_data.get("metrics"))
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ offsets = _table_time_offsets(form_data, query)
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if offsets:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
Review Comment:
Fixed in `74cad83129d2fde30b08dac585523da0087198bb`: Table comparisons use ordinary metrics only, retaining distinct percentage-only metric and contribution columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4189291740
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,795 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
Review Comment:
A labels-only update of a cached Bullet preview drops inherited `extra_form_data`/`extra_filters`, so a preview restricted to Region = EU and Last year is recached without those predicates. Could presentation-only updates preserve this native query context rather than discard it through the allowlist?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,795 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Retain final output sorts and rebind inherited metric expressions."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs = {
+ label
+ for column in new_form_data.get("groupby") or []
+ if (label := _column_label(column)) is not None
+ }
+ metrics = new_form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [new_form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ outputs.update(metric_outputs)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if isinstance(label, str) and label not in outputs:
Review Comment:
For a saved Bullet displaying `sales` by `region` but sorted by the valid saved metric `profit`, a labels-only update removes `orderby: [["profit", false]]` here, potentially changing the top category selected by `row_limit: 1`. Could omission preserve independent ranking metrics instead of pruning every sorter absent from the displayed outputs?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,610 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if type(value) in {list, tuple, dict}:
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(
+ {"queries": [{"data": [{"value": value}]}]}, temporal_json_numbers=True
+ )
+ if failure is not None or data is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} "
+ "has an invalid or unbounded container value"
+ )
+ normalized = data[0][0]["value"]
+ reason = None
+ elif _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if value_type in {list, dict}:
+ text = _bullet_container_category_text(normalized, dimension, row_index)
+ elif normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_container_category_text(value: Any, dimension: str, row_index: int) -> str:
+ """Stringify validated containers like JavaScript with a bounded text budget."""
+ from superset.mcp_service.chart.query_result import _bounded_utf8_length
+
+ if type(value) is dict:
+ return "[object Object]"
+ parts: list[str] = []
+ size = 0
+ for index, item in enumerate(value):
+ if type(item) in {list, dict}:
+ text = _bullet_container_category_text(item, dimension, row_index)
+ else:
+ text = (
+ ""
+ if item is None
+ else _bullet_category_value(item, dimension, row_index)[1]
+ )
+ text_size = _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES)
+ size += (text_size if text_size is not None else _MAX_BULLET_TEXT_BYTES + 1) + (
+ index > 0
+ )
+ if size > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ parts.append(text)
+ return ",".join(parts)
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ from superset.utils.number_format import D3_FORMAT_RE
+
+ # Specifier length does not bound its requested output precision. Check the
+ # parsed precision before the formatter can allocate or round any value.
+ match = D3_FORMAT_RE.match(format_)
+ if match and match.group(8) is not None and int(match.group(8)) > 20:
+ raise BulletOutputError(
+ "Bullet number format precision must not exceed 20",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet number format {format_!r} is unsupported by previews",
+ error_type="UnsupportedFormat",
+ ) from ex
+ return format_
+
+
+def _format_bullet_number(format_: str, value: float) -> str:
+ """Format finite Bullet values, including the full binary-float range."""
+ from superset.utils.number_format import format_numeric
+
+ try:
+ # Binary64 has at most 309 integer digits. Leave room for the bounded
+ # fractional precision, percent scaling, and a rounding carry without
+ # changing the caller's Decimal context.
+ with localcontext() as context:
+ context.prec = max(context.prec, 334)
+ return format_numeric(format_, value)
+ except OverflowError:
+ # SMART_NUMBER's significant-digit rounding can overflow a finite float
+ # near DBL_MAX. Scientific repr remains deterministic and informative.
+ if format_ in {"SMART_NUMBER", "SMART_NUMBER_SIGNED"} and math.isfinite(value):
+ prefix = "+" if format_ == "SMART_NUMBER_SIGNED" and value > 0 else ""
+ return prefix + repr(value)
+ raise
+
+
+def _containing_bullet_range_label(
+ measure: float, ranges: list[float], labels: list[str]
+) -> str | None:
+ """Match the frontend's labelled containing-range tooltip selection."""
+ ascending = sorted(
+ (
+ (value, labels[index] if index < len(labels) else "")
+ for index, value in enumerate(ranges)
+ ),
+ key=lambda entry: entry[0],
+ )
+ for threshold, label in ascending:
+ if measure <= threshold:
+ return label or None
+ if ascending and ascending[-1][1]:
+ return f"> {ascending[-1][1]}"
+ return None
+
+
+def resolve_bullet_render_model( # noqa: C901
+ data: List[Dict[str, Any]],
+ form_data: Dict[str, Any],
+ *,
+ validate_format: bool = True,
+) -> BulletRenderModel:
+ """Resolve Bullet rows; check preview formatter support only when rendering."""
+ if type(data) is not list:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ for row_index in range(list.__len__(data)):
+ row = list.__getitem__(data, row_index)
+ if type(row) is not dict:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ if dict.__len__(row) > _MAX_BULLET_FIELDS:
+ raise BulletOutputError("Bullet query row exceeds the field limit")
+ for key in dict.keys(row):
+ if type(key) is not str:
+ raise BulletOutputError("Bullet query row keys must be strings")
+ if len(key) > _MAX_BULLET_FIELD_BYTES:
+ raise BulletOutputError("Bullet query row key exceeds the size limit")
+
+ if type(form_data) is not dict:
+ raise BulletOutputError("Bullet form data must be an object")
+
+ metric_label = _form_metric_label(dict.get(form_data, "metric"))
+ if not metric_label:
+ raise BulletOutputError("Bullet metric has no declared result alias")
+ raw_groupby = dict.get(form_data, "groupby")
+ if raw_groupby is None:
+ raw_groupby = []
+ if type(raw_groupby) is not list:
Review Comment:
Saved Bullet groupings that render in Explore now fail MCP data reads/exports and previews: a scalar `groupby: "Region"` is rejected here, while an unlabeled SQL dimension loses its `sqlExpression` result-label fallback. Could hierarchy normalization and label lookup follow the native `ensureIsArray`/`getColumnLabel` contract?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1929 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
Review Comment:
This converts date cells to strings before `get_chart_data` chooses its export format, so an Excel export that previously contained date cells now contains General-format text, breaking date arithmetic and filtering; datetimes and durations are similarly affected. Could temporal objects remain available to the Excel writer while JSON projection happens separately?
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -154,6 +168,17 @@ def _compute_effective_force(request: GetChartDataRequest) -> bool:
return request.force_refresh or not request.use_cache
+def _download_form_data(form_data: dict[str, Any], request: Any) -> dict[str, Any]:
+ """Return form_data without UI page sizing for CSV/Excel downloads.
+
+ Server-paginated tables size a query to the UI page; a download keeps the
+ caller's limit instead of shrinking to that page.
+ """
+ if request.format in {"csv", "excel"} and form_data.get("server_pagination"):
Review Comment:
Removing this bypass would leave the export tests passing because they supply result rows, even though a server-paginated Table would again export only its 25-row UI page instead of the requested 1,000 rows. Could saved-fallback and cached `get_chart_data` regressions retain the real query builder and assert a 1,000-row main query plus more than 25 exported rows for CSV/Excel?
##########
superset/mcp_service/chart/compile.py:
##########
@@ -284,6 +331,459 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ # Match QueryObject's guarded legacy saved-metric normalization.
+ if not ({"sqlExpression", "aggregate", "column"} & value.keys()):
+ label = value.get("label")
+ if isinstance(label, str) and label:
+ return "saved_metric", label
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
+ viz_type = form_data.get("viz_type")
+ if strict_all_form_refs and (
+ viz_type == "mixed_timeseries"
+ or (
+ isinstance(viz_type, str)
+ and (
+ viz_type.startswith("echarts_timeseries") or viz_type == "echarts_area"
+ )
+ )
+ ):
+ if (raw_x_axis := form_data.get("x_axis")) is not None and (
+ error := column_error(raw_x_axis, "form-data x_axis column")
+ ):
+ return error
+ metric_fields = [
+ "metrics",
+ "size",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ ]
+ if viz_type == "mixed_timeseries":
+ metric_fields.extend(
+ [
+ "metrics_b",
+ "size_b",
+ "timeseries_limit_metric_b",
+ "series_limit_metric_b",
+ ]
+ )
+ for field_name in metric_fields:
+ raw_value = form_data.get(field_name)
+ values = raw_value if isinstance(raw_value, list) else [raw_value]
+ for value in values:
+ if value is not None and (
+ error := metric_error(value, f"form-data {field_name} metric")
+ ):
+ return error
+
+ layer_suffixes = ("", "_b") if viz_type == "mixed_timeseries" else ("",)
+ for suffix in layer_suffixes:
+ sort_field = f"x_axis_sort{suffix}"
+ if sort_field not in form_data or form_data.get(sort_field) is None:
+ continue
+ x_axis = form_data.get(f"x_axis{suffix}", form_data.get("x_axis"))
+ allowed_labels: set[str] = set()
+ if x_axis_label := _native_column_label(x_axis):
+ allowed_labels.add(x_axis_label)
+ raw_metrics = form_data.get(f"metrics{suffix}")
+ for metric in raw_metrics if isinstance(raw_metrics, list) else []:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ raw_limit_metric = form_data.get(f"timeseries_limit_metric{suffix}")
+ limit_metrics = (
+ raw_limit_metric
+ if isinstance(raw_limit_metric, list)
+ else [raw_limit_metric]
+ )
+ for metric in limit_metrics:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ sort_value = form_data[sort_field]
+ if not isinstance(sort_value, str) or sort_value not in allowed_labels:
+ return _native_validation_error(sort_field, repr(sort_value)[:200])
+
+ if (
+ strict_all_form_refs
+ and isinstance(viz_type, str)
+ and viz_type.startswith("deck_")
+ ):
+ # Deck layers store most query roles outside common columns/metrics.
+ # Validate every renderer-consumed raw control as well as the generated
+ # QueryObject so a dataset-only rebind cannot hide or discard a stale
+ # tooltip, cross-filter, spatial, path, or metric reference.
+ for spatial_field in ("spatial", "start_spatial", "end_spatial"):
+ spatial = form_data.get(spatial_field)
+ if spatial is None:
+ continue
+ if not isinstance(spatial, dict):
+ return _native_validation_error(
+ f"form-data {spatial_field}", repr(spatial)[:200]
+ )
+ spatial_type = spatial.get("type")
+ if not isinstance(spatial_type, str):
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ role_fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if role_fields is None:
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ for role_field in role_fields:
+ spatial_value = spatial.get(role_field)
+ if spatial_value is None:
+ return _native_validation_error(
+ f"form-data {spatial_field}.{role_field} column", "missing"
+ )
+ if error := column_error(
+ spatial_value, f"form-data {spatial_field}.{role_field} column"
+ ):
+ return error
+
+ for field_name in (
+ "line_column",
+ "geojson",
+ "dimension",
+ "cross_filter_column",
+ ):
+ column_value = form_data.get(field_name)
+ if column_value is not None and (
+ error := column_error(column_value, f"form-data {field_name} column")
+ ):
+ return error
+
+ tooltip_contents = form_data.get("tooltip_contents")
+ if tooltip_contents is not None and not isinstance(tooltip_contents, list):
+ return _native_validation_error(
+ "form-data tooltip_contents", repr(tooltip_contents)[:200]
+ )
+ for index, item in enumerate(tooltip_contents or []):
+ tooltip_value: Any = None
+ if isinstance(item, str):
+ tooltip_value = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ tooltip_value = item.get("column_name")
+ if tooltip_value is not None and (
+ error := column_error(
+ tooltip_value, f"form-data tooltip_contents[{index}] column"
+ )
+ ):
+ return error
+
+ metric_values: list[tuple[str, Any]] = []
+ if viz_type not in {"deck_geojson", "deck_polygon"}:
+ for field_name in ("metrics", "metric", "size"):
+ raw_deck_metrics = form_data.get(field_name)
+ deck_metrics = (
+ raw_deck_metrics
+ if isinstance(raw_deck_metrics, list)
+ else [raw_deck_metrics]
+ )
+ metric_values.extend(
+ (f"form-data {field_name} metric", deck_metric)
+ for deck_metric in deck_metrics
+ if deck_metric is not None
+ )
+ if viz_type == "deck_polygon" and form_data.get("metric") is not None:
+ metric_values.append(("form-data metric metric", form_data.get("metric")))
+ fixed_metric_fields = (
+ ("point_radius_fixed",)
+ if viz_type in {"deck_scatter", "deck_polygon"}
+ else ()
+ ) + (("line_width",) if viz_type == "deck_path" else ())
+ for field_name in fixed_metric_fields:
+ fixed_value = form_data.get(field_name)
+ deck_metric: Any = (
+ fixed_value
+ if (
+ isinstance(fixed_value, str)
+ and fixed_value
+ and viz_type != "deck_polygon"
+ )
+ else None
+ )
+ if isinstance(fixed_value, dict) and fixed_value.get("type") == "metric":
+ deck_metric = fixed_value.get("value")
+ if deck_metric is not None:
+ metric_values.append((f"form-data {field_name} metric", deck_metric))
+ if viz_type == "deck_path" and form_data.get("breakpoint_metric") is not None:
+ metric_values.append(
+ (
+ "form-data breakpoint_metric metric",
+ form_data.get("breakpoint_metric"),
+ )
+ )
+ for role, deck_metric in metric_values:
+ if error := metric_error(deck_metric, role):
+ return error
+
+ for filter_ in form_data.get("adhoc_filters") or []:
+ if not isinstance(filter_, dict) or filter_.get("expressionType") != "SIMPLE":
+ continue
+ if not strict_all_form_refs and _is_inert_adhoc_filter(filter_):
+ continue
+ subject = filter_.get("subject")
+ clause = str(filter_.get("clause") or "WHERE").upper()
+ if clause == "HAVING" and isinstance(subject, str):
+ metric_matches = (
+ [subject]
+ if subject in saved_metrics
+ else [
+ name
+ for name in saved_metrics
+ if name.casefold() == subject.casefold()
+ ]
+ )
+ if len(metric_matches) == 1:
+ continue
+ if subject is not None and (
+ error := column_error(subject, "form-data filter column")
+ ):
+ return error
+ if filter_.get("operator") == "TEMPORAL_RANGE" and isinstance(subject, str):
+ try:
+ temporal = resolve_dataset_column(subject, dataset_context)
+ except ValueError:
+ temporal = None
+ if temporal is not None and not temporal.get("is_temporal", False):
+ return _native_validation_error("temporal filter column", subject)
+
+ # temporal_columns_lookup describes the entire datasource, not selected
+ # roles. The physical form/query column checks validate selected references.
+
+ for query_index, query in enumerate(queries, 1):
+ metric_labels: set[str] = set()
+ for column in query.get("columns") or []:
+ if error := column_error(column, f"query {query_index} column"):
+ return error
+ for column in query.get("series_columns") or []:
+ if error := column_error(column, f"query {query_index} series column"):
+ return error
+ for column in query.get("groupby") or []:
+ if error := column_error(column, f"query {query_index} groupby column"):
+ return error
+ selected_column_labels = {
+ label
+ for column in query.get("columns") or []
+ if (label := _native_column_label(column)) is not None
+ }
+ for level in query.get("grouping_sets") or []:
+ for column in level:
+ # Grouping sets reference selected logical outputs, including
+ # Custom SQL labels. Their source columns were checked above.
+ if isinstance(column, str) and column in selected_column_labels:
+ continue
+ if error := column_error(
+ column, f"query {query_index} grouping-set column"
+ ):
+ return error
+
+ metrics = query.get("metrics") or []
+ for metric in metrics:
+ if label := _metric_label_for_validation(metric):
+ metric_labels.add(label)
+ if error := metric_error(metric, f"query {query_index} metric"):
+ return error
+
+ granularity = query.get("granularity")
+ if granularity:
+ if error := column_error(granularity, "temporal column"):
+ return error
+ try:
+ temporal = resolve_dataset_column(granularity, dataset_context)
+ except ValueError:
+ temporal = None
+ if (
+ (query.get("extras") or {}).get("time_grain_sqla")
+ and temporal is not None
+ and not temporal.get("is_temporal", False)
+ ):
+ return _native_validation_error("temporal column", granularity)
+
+ for filter_ in query.get("filters") or []:
+ if not isinstance(filter_, dict):
+ return _native_validation_error("filter", repr(filter_)[:200])
+ column = filter_.get("col")
+ if isinstance(column, str) and column in metric_labels:
+ continue
+ if isinstance(column, str) and any(
+ name.casefold() == column.casefold() for name in saved_metrics
+ ):
+ continue
+ if column is not None and (
+ error := column_error(column, f"query {query_index} filter column")
+ ):
+ return error
+
+ for order in query.get("orderby") or []:
+ if not isinstance(order, (list, tuple)) or len(order) != 2:
+ return _native_validation_error("ordering", repr(order)[:200])
+ target = order[0]
+ target_label = _metric_label_for_validation(target)
+ if target in metrics or (target_label and target_label in metric_labels):
+ continue
+ if isinstance(target, str) and target in metric_labels:
+ continue
+ # Native ordering can use a metric that is not displayed. Resolve
+ # saved names with the same exact/case-folded rules as other metrics,
+ # and validate adhoc metric columns rather than their output labels.
+ if (
+ isinstance(target, str)
+ and any(name.casefold() == target.casefold() for name in saved_metrics)
+ ) or (isinstance(target, dict) and _native_metric_ref(target) is not None):
+ if error := metric_error(
+ target, f"query {query_index} ordering metric"
+ ):
+ return error
Review Comment:
A saved Bullet grouped by SQL expression `UPPER(region)` labeled `RegionUpper`, with `orderby: [["RegionUpper", true]]`, now fails even a labels-only update because this treats the selected output alias as a missing physical column. Could ordering targets also resolve against `selected_column_labels`, as the native SQL query path does?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,610 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if type(value) in {list, tuple, dict}:
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(
+ {"queries": [{"data": [{"value": value}]}]}, temporal_json_numbers=True
+ )
+ if failure is not None or data is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} "
+ "has an invalid or unbounded container value"
+ )
+ normalized = data[0][0]["value"]
+ reason = None
+ elif _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if value_type in {list, dict}:
+ text = _bullet_container_category_text(normalized, dimension, row_index)
+ elif normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_container_category_text(value: Any, dimension: str, row_index: int) -> str:
+ """Stringify validated containers like JavaScript with a bounded text budget."""
+ from superset.mcp_service.chart.query_result import _bounded_utf8_length
+
+ if type(value) is dict:
+ return "[object Object]"
+ parts: list[str] = []
+ size = 0
+ for index, item in enumerate(value):
+ if type(item) in {list, dict}:
+ text = _bullet_container_category_text(item, dimension, row_index)
+ else:
+ text = (
+ ""
+ if item is None
+ else _bullet_category_value(item, dimension, row_index)[1]
+ )
+ text_size = _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES)
+ size += (text_size if text_size is not None else _MAX_BULLET_TEXT_BYTES + 1) + (
+ index > 0
+ )
+ if size > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ parts.append(text)
+ return ",".join(parts)
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ from superset.utils.number_format import D3_FORMAT_RE
+
+ # Specifier length does not bound its requested output precision. Check the
+ # parsed precision before the formatter can allocate or round any value.
+ match = D3_FORMAT_RE.match(format_)
+ if match and match.group(8) is not None and int(match.group(8)) > 20:
+ raise BulletOutputError(
+ "Bullet number format precision must not exceed 20",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet number format {format_!r} is unsupported by previews",
+ error_type="UnsupportedFormat",
+ ) from ex
+ return format_
+
+
+def _format_bullet_number(format_: str, value: float) -> str:
+ """Format finite Bullet values, including the full binary-float range."""
+ from superset.utils.number_format import format_numeric
+
+ try:
+ # Binary64 has at most 309 integer digits. Leave room for the bounded
+ # fractional precision, percent scaling, and a rounding carry without
+ # changing the caller's Decimal context.
+ with localcontext() as context:
+ context.prec = max(context.prec, 334)
+ return format_numeric(format_, value)
+ except OverflowError:
+ # SMART_NUMBER's significant-digit rounding can overflow a finite float
+ # near DBL_MAX. Scientific repr remains deterministic and informative.
+ if format_ in {"SMART_NUMBER", "SMART_NUMBER_SIGNED"} and math.isfinite(value):
+ prefix = "+" if format_ == "SMART_NUMBER_SIGNED" and value > 0 else ""
+ return prefix + repr(value)
+ raise
+
+
+def _containing_bullet_range_label(
+ measure: float, ranges: list[float], labels: list[str]
+) -> str | None:
+ """Match the frontend's labelled containing-range tooltip selection."""
+ ascending = sorted(
+ (
+ (value, labels[index] if index < len(labels) else "")
+ for index, value in enumerate(ranges)
+ ),
+ key=lambda entry: entry[0],
+ )
+ for threshold, label in ascending:
+ if measure <= threshold:
+ return label or None
+ if ascending and ascending[-1][1]:
+ return f"> {ascending[-1][1]}"
+ return None
+
+
+def resolve_bullet_render_model( # noqa: C901
+ data: List[Dict[str, Any]],
+ form_data: Dict[str, Any],
+ *,
+ validate_format: bool = True,
+) -> BulletRenderModel:
+ """Resolve Bullet rows; check preview formatter support only when rendering."""
+ if type(data) is not list:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ for row_index in range(list.__len__(data)):
+ row = list.__getitem__(data, row_index)
+ if type(row) is not dict:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ if dict.__len__(row) > _MAX_BULLET_FIELDS:
+ raise BulletOutputError("Bullet query row exceeds the field limit")
+ for key in dict.keys(row):
+ if type(key) is not str:
+ raise BulletOutputError("Bullet query row keys must be strings")
+ if len(key) > _MAX_BULLET_FIELD_BYTES:
+ raise BulletOutputError("Bullet query row key exceeds the size limit")
+
+ if type(form_data) is not dict:
+ raise BulletOutputError("Bullet form data must be an object")
+
+ metric_label = _form_metric_label(dict.get(form_data, "metric"))
+ if not metric_label:
+ raise BulletOutputError("Bullet metric has no declared result alias")
+ raw_groupby = dict.get(form_data, "groupby")
+ if raw_groupby is None:
+ raw_groupby = []
+ if type(raw_groupby) is not list:
+ raise BulletOutputError("Bullet dimensions must be an array")
+ dimension_labels = [
+ _form_column_label(list.__getitem__(raw_groupby, index))
+ for index in range(list.__len__(raw_groupby))
+ ]
+ if any(not label for label in dimension_labels):
+ raise BulletOutputError("Bullet dimension has no declared result alias")
+
+ if data:
+ first_row = list.__getitem__(data, 0)
+ metric_field = _require_result_field(metric_label, first_row, "metric")
+ dimensions = [
+ _require_result_field(label, first_row, "dimension")
+ for label in dimension_labels
+ ]
+ else:
+ # The frontend accepts empty results. Ungrouped charts retain one
+ # zero-valued measure; grouped charts retain the declared roles but no
+ # categories or rows are fabricated.
+ metric_field = metric_label
+ dimensions = [label for label in dimension_labels if label is not None]
+
+ measures: list[float] = []
+ copied_rows: list[dict[str, Any]] = []
+ for index in range(list.__len__(data)):
+ row = list.__getitem__(data, index)
+ row_metric_field = _require_result_field(
+ metric_label, row, f"metric row {index}"
+ )
+ measure = _bullet_number(
+ dict.__getitem__(row, row_metric_field), index, metric_field
+ )
+ # Reserve every exact output key so the internal Vega category alias
+ # cannot collide with an unselected result field. Unselected values are
+ # deliberately replaced with None rather than converted or serialized.
+ copied: dict[str, Any] = dict.fromkeys(dict.keys(row))
+ copied[metric_field] = measure
+ for label, dimension in zip(dimension_labels, dimensions, strict=True):
+ row_dimension = _require_result_field(label, row, f"dimension row {index}")
+ dimension_value, _ = _bullet_category_value(
+ dict.__getitem__(row, row_dimension), dimension, index
+ )
+ copied[dimension] = dimension_value
+ copied_rows.append(copied)
+ measures.append(measure)
+
+ # The frontend validates/coerces the whole result array but renders only
+ # the first row for an ungrouped aggregate.
+ if not dimensions:
+ copied_rows = copied_rows[:1]
+ measures = measures[:1]
+ if not copied_rows:
+ copied_rows = [{metric_field: 0.0}]
+ measures = [0.0]
+
+ ranges = _bullet_numeric_control_tokens(dict.get(form_data, "ranges"), "ranges")
+ if not ranges:
+ # Match Bullet/transformProps.ts: the largest measure drives one
+ # qualitative band whose upper threshold is 110% of that measure.
+ ranges = [0.0, max(measures, default=0.0) * 1.1]
+ markers = _bullet_numeric_control_tokens(dict.get(form_data, "markers"), "markers")
+ marker_lines = _bullet_numeric_control_tokens(
+ dict.get(form_data, "marker_lines"), "marker lines"
+ )
+ all_numbers = [*measures, *ranges, *markers, *marker_lines]
+ if any(not math.isfinite(value) for value in all_numbers):
+ raise BulletOutputError("Bullet presentation values must be finite")
+
+ show_labels = dict.get(form_data, "show_labels", False)
+ show_legend = dict.get(form_data, "show_legend", False)
+ if type(show_labels) is not bool or type(show_legend) is not bool:
+ raise BulletOutputError("Bullet label and legend controls must be booleans")
+
+ range_labels = _bullet_string_tokens(dict.get(form_data, "range_labels"))
+ marker_labels = _bullet_string_tokens(dict.get(form_data, "marker_labels"))
+ marker_line_labels = _bullet_string_tokens(
+ dict.get(form_data, "marker_line_labels")
+ )
+ return BulletRenderModel(
+ rows=copied_rows,
+ metric_field=metric_field,
+ dimensions=dimensions,
+ measures=measures,
+ ranges=ranges,
+ range_labels=range_labels,
+ markers=markers,
+ marker_labels=marker_labels,
+ marker_lines=marker_lines,
+ marker_line_labels=marker_line_labels,
+ y_axis_format=(
+ _validate_bullet_format(
+ dict.get(form_data, "y_axis_format", "SMART_NUMBER"), all_numbers
+ )
+ if validate_format
+ else "SMART_NUMBER"
+ ),
+ show_labels=show_labels,
+ show_legend=show_legend,
+ )
+
+
+def _generate_ascii_bullet_chart(
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> str:
+ """Generate a horizontal Bullet preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+ if model.dimensions and not model.rows:
+ return "No data available for grouped Bullet chart"
+ extent = (
+ max(
+ [abs(value) for value in [*model.measures, *model.ranges, *model.markers]],
+ default=1,
+ )
+ or 1
+ )
+ lines = [f"ASCII Bullet Chart — {model.metric_field}", "=" * 60]
+ for row_index, (row, value) in enumerate(
+ zip(model.rows[:10], model.measures[:10], strict=True)
+ ):
+ category = ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ category = category or "Measure"
+ width = round(abs(value) / extent * 32)
+ bar = "█" * width
+ formatted = _format_bullet_number(model.y_axis_format, value)
+ lines.append(f"{category[:20]:>20} |{bar:<32} {formatted}")
+
+ def labeled(values: list[float], labels: list[str], prefix: str) -> list[str]:
+ return [
+ f"{labels[index] if index < len(labels) and labels[index] else prefix}: "
+ f"{_format_bullet_number(model.y_axis_format, value)}"
+ for index, value in enumerate(values)
+ ]
+
+ if model.show_labels or model.show_legend or model.marker_lines:
+ lines.append("Key:")
+ if model.show_labels or model.show_legend:
Review Comment:
With the native defaults `show_labels=false` and `show_legend=false`, the ASCII preview omits ranges and markers entirely: a measure of 75 with range 100 looks identical for goal 80 and goal 60, hiding whether it misses or exceeds the goal. Could comparison targets remain visible independently of label and legend toggles?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4226256911
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,137 @@ def fallback_vega_lite_preview(
return None
+def _xy_pivot_x_type(values: list[Any]) -> str:
+ """Infer the Vega-Lite x type from every x value, not a character scan.
+
+ The renderer has no column metadata, so text is temporal only when each
+ value parses as an ISO date or datetime; labels such as ``New York`` stay
+ nominal instead of becoming unparseable dates.
+ """
+ present = [value for value in values if value is not None]
+ if not present:
+ return "nominal"
+ if all(
+ isinstance(value, (int, float)) and not isinstance(value, bool)
+ for value in present
+ ):
+ return "quantitative"
+ if all(
+ isinstance(value, (date, datetime))
+ or (isinstance(value, str) and _gantt_temporal_value(value) is not None)
Review Comment:
Confirmed: `datetime.fromisoformat` accepts basic-format (`20250101`, `2025-01-01T000000`) and week dates (`2025-W01-1`), and `Date.parse` returns NaN for all of them. Fixed in 4d0ef0f964051990f0e86e9583c04a8d8eccc604: `_xy_pivot_x_type` marks text temporal only when every value fully matches the extended ISO form (`YYYY-MM-DD` with optional `T`/space time, fractional seconds and `Z`/`±HH:MM` offset) and still parses; anything else falls back to `nominal`. `test_xy_preview_x_type_requires_parseable_dates` adds compact, mixed, week-date, basic-time and invalid-month cases (4 of them fail without the fix), plus extended forms that stay temporal.
##########
superset/mcp_service/semantic_layer/tool/get_table.py:
##########
@@ -509,33 +563,38 @@ async def _run_get_table_query(
use_cache=request.use_cache,
force=request.force_refresh,
)
+ extracted = _extract_table_query_result(result)
+ if isinstance(extracted, SemanticLayerError):
+ return extracted
+ query_result, data, raw_columns, coltypes = extracted
query_duration_ms = int((time.time() - start_time) * 1000)
- if not result or "queries" not in result or not result["queries"]:
- return SemanticLayerError.create(
- error="Query returned no results.",
- error_type="EmptyQuery",
- )
-
await ctx.report_progress(5, 5, "Formatting results")
- query_result = result["queries"][0]
response = _build_response(
request,
is_builtin,
resolved.display_name,
query_result,
+ data,
+ raw_columns,
+ coltypes,
query_duration_ms,
resolved.warnings,
resolved.temporal_columns,
resolved.valid_grains,
)
+ if response_failure := response_json_failure(response):
Review Comment:
Added in 4d0ef0f964051990f0e86e9583c04a8d8eccc604: `test_get_table_rejects_response_amplified_by_column_profiling` sends 255 rows, each with a distinct ~64 KiB `category` cell. The test asserts that `query_result_data` accepts the source result, so source validation passes. It then wraps `response_json_failure` to confirm the post-format guard is reached and returns the failure, and asserts a `MalformedQueryResult` byte-limit error with no `data`/`columns`. With the guard block disabled, the test fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4243794597
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -3985,6 +4104,487 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ # Saved query roles are internal sort context, never typed authoring input.
+ # Keeping them out of dimensions preserves omission and native SQL shapes.
+ _inherited_groupby: list[str | dict[str, Any]] | None = PrivateAttr(default=None)
+
+ @property
+ def order_dimensions(self) -> Sequence[ColumnRef | str | dict[str, Any]]:
+ """Return authored dimensions or the inherited native sort hierarchy."""
+ if self.dimensions is None and self._inherited_groupby is not None:
+ return self._inherited_groupby
+ return self.dimensions or []
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
Review Comment:
`_adapt_native_metric` gives an unlabeled SQL metric the label `sqlExpression`, but here only `label` or `metric_name` is read from a metric object. A native Bullet with `metric: {expressionType: "SQL", sqlExpression: "SUM(revenue)"}` and `orderby: [[<same metric object>, false]]` therefore hits the "needs a column or metric label" error even though the metric itself validates. Should this fall back to `sqlExpression` the same way the metric adapter does?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5497428373
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/43770?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `16.49049%` with `395 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 66.00%. Comparing base ([`4395640`](https://app.codecov.io/gh/apache/superset/commit/439564041f54cafbc17f2f995c682c65b8bbbaab?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6411173`](https://app.codecov.io/gh/apache/superset/commit/6411173bebabf11e12f1f6ae8983c7dd137fe14e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/43770?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/mcp\_service/chart/schemas.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Fschemas.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvc2NoZW1hcy5weQ==) | 22.51% | [148 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/mcp\_service/chart/preview\_utils.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Fpreview_utils.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvcHJldmlld191dGlscy5weQ==) | 6.79% | [96 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/mcp\_service/chart/plugins/bullet.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Fplugins%2Fbullet.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvcGx1Z2lucy9idWxsZXQucHk=) | 21.64% | [76 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/mcp\_service/chart/chart\_utils.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Fchart_utils.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvY2hhcnRfdXRpbHMucHk=) | 9.52% | [38 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...perset/mcp\_service/chart/tool/get\_chart\_preview.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Ftool%2Fget_chart_preview.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvdG9vbC9nZXRfY2hhcnRfcHJldmlldy5weQ==) | 3.22% | [30 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/mcp\_service/chart/chart\_helpers.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Fchart_helpers.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvY2hhcnRfaGVscGVycy5weQ==) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/mcp\_service/chart/tool/update\_chart.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Ftool%2Fupdate_chart.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvdG9vbC91cGRhdGVfY2hhcnQucHk=) | 0.00% | [2 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...erset/mcp\_service/chart/resources/chart\_configs.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Fresources%2Fchart_configs.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvcmVzb3VyY2VzL2NoYXJ0X2NvbmZpZ3MucHk=) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [superset/mcp\_service/chart/tool/get\_chart\_data.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Ftool%2Fget_chart_data.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvdG9vbC9nZXRfY2hhcnRfZGF0YS5weQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...set/mcp\_service/chart/tool/update\_chart\_preview.py](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&filepath=superset%2Fmcp_service%2Fchart%2Ftool%2Fupdate_chart_preview.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvbWNwX3NlcnZpY2UvY2hhcnQvdG9vbC91cGRhdGVfY2hhcnRfcHJldmlldy5weQ==) | 0.00% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/43770?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #43770 +/- ##
==========================================
+ Coverage 57.45% 66.00% +8.54%
==========================================
Files 2889 2890 +1
Lines 166762 167232 +470
Branches 38604 38733 +129
==========================================
+ Hits 95818 110377 +14559
+ Misses 70047 54617 -15430
- Partials 897 2238 +1341
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/43770/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/43770/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.70% <16.49%> (-0.13%)` | :arrow_down: |
| [presto](https://app.codecov.io/gh/apache/superset/pull/43770/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.59% <16.49%> (-0.14%)` | :arrow_down: |
| [python](https://app.codecov.io/gh/apache/superset/pull/43770/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.08% <16.49%> (+17.30%)` | :arrow_up: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/43770/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.95% <16.49%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/43770?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3906292876
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -469,11 +567,126 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _generate_bullet_vega_lite_preview(
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview | None:
+ """Build a horizontal layered preview faithful to Bullet result roles."""
+ metric_field, dimensions = _bullet_result_roles(data, form_data)
+ if metric_field is None:
+ return None
+
+ category_field = "__mcp_bullet_category"
+ values = []
+ for index, row in enumerate(data):
+ copied = dict(row)
+ copied[category_field] = (
+ ", ".join(str(row.get(field, "")) for field in dimensions)
+ if dimensions
+ else str(index + 1)
+ )
+ values.append(copied)
+
+ y_encoding = {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(dimensions) if dimensions else None,
+ "sort": None,
+ }
+ tooltip = [
+ *({"field": field, "type": "nominal"} for field in dimensions),
+ {"field": metric_field, "type": "quantitative"},
+ ]
+ layers: list[dict[str, Any]] = []
+ for index, threshold in enumerate(
+ sorted(_bullet_numeric_tokens(form_data.get("ranges")), reverse=True)
+ ):
+ layers.append(
+ {
+ "mark": {
+ "type": "rect",
+ "opacity": max(0.08, 0.28 - index * 0.04),
+ },
+ "encoding": {
+ "x": {"datum": 0, "type": "quantitative"},
+ "x2": {"datum": threshold},
Review Comment:
**Suggestion:** Range rectangles have no `y` encoding, so grouped Bullet previews render threshold backgrounds across the chart instead of one background band per category row. [logic error]
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Sometimes`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=55dc227cbd314162883dc677be6e31af&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=55dc227cbd314162883dc677be6e31af&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/mcp_service/chart/preview_utils.py
**Line:** 628:630
**Comment:**
*Logic Error: Range rectangles have no `y` encoding, so grouped Bullet previews render threshold backgrounds across the chart instead of one background band per category row.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=3a5c37f928269ffae84d050c2b06f74b097cbb4bec371e95de2a3da7a5ef1c99&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=3a5c37f928269ffae84d050c2b06f74b097cbb4bec371e95de2a3da7a5ef1c99&reaction=dislike'>👎</a>
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -323,6 +325,102 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if isinstance(metric, str):
+ return metric
+ if not isinstance(metric, dict):
+ return None
+ if label := metric.get("label"):
+ return label if isinstance(label, str) else None
+ column = metric.get("column")
+ column_name = column.get("column_name") if isinstance(column, dict) else column
+ aggregate = metric.get("aggregate")
+ if isinstance(column_name, str) and isinstance(aggregate, str):
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if isinstance(column, str):
+ return column
+ if not isinstance(column, dict):
+ return None
+ for key in ("label", "column_name"):
+ if isinstance(value := column.get(key), str) and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve a query result field by exact match, then unambiguous casefold."""
+ if label is None:
+ return None
+ if label in row:
+ return label
+ matches = [field for field in row if field.casefold() == label.casefold()]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _bullet_result_roles(
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> tuple[str | None, list[str]]:
+ """Resolve Bullet metric and full category hierarchy in query output."""
+ if not data:
+ return None, []
+ first_row = data[0]
+ metric_field = _canonical_result_field(
+ _form_metric_label(form_data.get("metric")), first_row
+ )
+ if metric_field is None:
+ metric_field = next(
+ (
+ field
+ for field, value in first_row.items()
+ if isinstance(value, (int, float)) and not _is_nan(value)
+ ),
+ None,
+ )
Review Comment:
**Suggestion:** When the metric label is absent from results, this fallback selects the first numeric field, so a numeric dimension can be rendered as the Bullet measure. [incorrect variable usage]
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Sometimes`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=85cb1d636aea49b6aa0862104d7ec879&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=85cb1d636aea49b6aa0862104d7ec879&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/mcp_service/chart/preview_utils.py
**Line:** 373:384
**Comment:**
*Incorrect Variable Usage: When the metric label is absent from results, this fallback selects the first numeric field, so a numeric dimension can be rendered as the Bullet measure.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=4ec059a2b99b8cfff1f87add7d89f0a34538bd807e5bcf1436c29c19186047a2&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=4ec059a2b99b8cfff1f87add7d89f0a34538bd807e5bcf1436c29c19186047a2&reaction=dislike'>👎</a>
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2014,6 +2016,385 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Typed contract for the ECharts Bullet visualization (viz_type ``bullet``).
+
+ Semantic field names are exposed to MCP clients while validation aliases and
+ the native adapter accept saved Explore ``form_data`` without weakening the
+ unknown-field checks that catch misspelled controls.
+ """
+
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description=(
+ "Numeric measure shown by each bullet bar. Use aggregate for a SIMPLE "
+ "metric, saved_metric=True for a dataset metric, or sql_expression "
+ "with a unique label."
+ ),
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Optional category hierarchy; the frontend renders one bullet row per "
+ "unique combination (native form_data: groupby). Omit to preserve a "
+ "saved hierarchy on update; pass [] to clear it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(
+ None,
+ description=(
+ "Structured WHERE filters. Native SIMPLE adhoc_filters are accepted; "
+ "free-form SQL filters are rejected."
+ ),
+ max_length=100,
+ )
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Optional Superset time range such as 'Last 30 days' or "
+ "'2025-01-01 : 2025-12-31'. Set temporal_column to choose its column."
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum grouped bullet rows returned by the query",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description=(
+ "Stable row ordering by a dimension name or by the metric's output "
+ "label/name. Native orderby pairs and order_by_cols JSON pairs are "
+ "accepted for saved-form-data round trips."
+ ),
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ legacy = re.fullmatch(
+ r"(sum|avg|min|max|count|count_distinct)__(.+)",
+ value,
+ flags=re.IGNORECASE,
+ )
+ if legacy:
+ return {
+ "name": legacy.group(2),
+ "aggregate": legacy.group(1).upper(),
+ }
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict) or "expressionType" not in value:
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ data.setdefault("temporal_column", subject)
+ if isinstance(comparator, str) and comparator.casefold() != "no filter":
+ data.setdefault("time_range", comparator)
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {"==": "=", "IS_NOT_NULL": "IS NOT NULL"}
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ if data.get("viz_type") == "bullet":
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ ):
+ data.pop(key, None)
+
+ marker_key = "_mcp_dashboard_time_filter_subject"
+ if marker := data.pop(marker_key, None):
+ if not isinstance(marker, str):
+ raise ValueError(f"{marker_key} must be a physical column name")
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
+ {"name": item} if isinstance(item, str) else item
+ for item in data[key]
Review Comment:
**Suggestion:** When both `dimensions` and `groupby` are present, the alias silently chooses `dimensions`, so conflicting saved and semantic hierarchies can update the chart with the wrong grouping. [api mismatch]
**Assessment:** 🟠 `Major` · 🔁 `Occurrence: Sometimes`
[](https://docs.codeant.ai/cli/resolve-pr-comments-skill) [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=1e04d58b3a2449c0b643a921cd6b3f39&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=1e04d58b3a2449c0b643a921cd6b3f39&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset)
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/mcp_service/chart/schemas.py
**Line:** 2265:2271
**Comment:**
*Api Mismatch: When both `dimensions` and `groupby` are present, the alias silently chooses `dimensions`, so conflicting saved and semantic hierarchies can update the chart with the wrong grouping.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
```
</details>
<a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=639e0106c120088ec00b7ae4097f4e2bd110c3164101d99d5cece0803b1c0d7b&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F43770&comment_hash=639e0106c120088ec00b7ae4097f4e2bd110c3164101d99d5cece0803b1c0d7b&reaction=dislike'>👎</a>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4224607635
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,847 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if isinstance(existing_viz_type, str) and existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+def _currency_form_value(value: CurrencyFormat | None) -> dict[str, str] | None:
+ """Return the native value for an explicitly supplied currency control."""
+ return value.to_form_data() if value is not None else None
+
+
+def _column_names(value: Sequence[ColumnRef] | None) -> list[str | None] | None:
+ """Return a native column-name list while retaining an explicit null."""
+ return [column.name for column in value] if value is not None else None
+
+
+def _table_sort_value(value: Sequence[str | SortByConfig] | None) -> list[str] | None:
+ """Return the native Table sort control for an explicit typed value."""
+ if value is None:
+ return None
+ return [
+ json.dumps(
+ [entry.column, entry.ascending]
+ if isinstance(entry, SortByConfig)
+ else [entry, False]
+ )
+ for entry in value
+ ]
+
+
+def _table_column_config_value(value: Any) -> dict[str, Any] | None:
+ """Return Table column config without losing an explicit null or empty map."""
+ if value is None:
+ return None
+ return {
+ label: column.model_dump(by_alias=True, exclude_unset=True)
+ for label, column in value.items()
+ }
+
+
+# Mappers intentionally omit optional controls so fresh charts use the frontend
+# defaults. During a same-viz update, however, an explicitly supplied false,
+# null, or empty value must block preservation of the saved native key. Keep the
+# typed-to-native relationship declarative so every update path shares it.
+_FormValueConverter = Callable[[Any], Any]
+_FormControlMap = dict[str, tuple[str, _FormValueConverter]]
+
+_COMMON_EXPLICIT_FORM_CONTROLS: _FormControlMap = {
+ "color_scheme": ("color_scheme", lambda value: value),
+ "currency_format": ("currency_format", _currency_form_value),
+ "show_value": ("show_value", lambda value: value),
+}
+
+_CHART_EXPLICIT_FORM_CONTROLS: dict[str, _FormControlMap] = {
+ "table": {
+ "sort_by": ("order_by_cols", _table_sort_value),
+ "column_config": ("column_config", _table_column_config_value),
+ },
+ "xy": {
+ "group_by": ("groupby", _column_names),
+ "series_limit": ("series_limit", lambda value: value),
+ "stacked": ("stack", lambda value: "Stack" if value else None),
+ "orientation": ("orientation", lambda value: value),
+ "legend_orientation": ("legendOrientation", lambda value: value),
+ "x_axis_time_format": ("x_axis_time_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "mixed_timeseries": {
+ "group_by": ("groupby", _column_names),
+ "group_by_secondary": ("groupby_b", _column_names),
+ "currency_format_secondary": (
+ "currency_format_secondary",
+ _currency_form_value,
+ ),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "waterfall": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "big_number": {
+ "subheader": ("subheader", lambda value: value),
+ "y_axis_format": ("y_axis_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "compare_lag": ("compare_lag", lambda value: value),
+ "time_format": ("time_format", lambda value: value),
+ "aggregation": ("aggregation", lambda value: value),
+ },
+ "handlebars": {
+ "style_template": ("styleTemplate", lambda value: value),
+ "columns": ("all_columns", _column_names),
+ "groupby": ("groupby", _column_names),
+ "metrics": ("metrics", _column_names),
+ },
+ "pivot_table": {
+ "date_format": ("date_format", lambda value: value),
+ },
+ "interactive_pivot": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "series_limit": ("series_limit", lambda value: value),
+ "date_format": ("date_format", lambda value: value),
+ "column_sort": ("colOrder", lambda value: value),
+ },
+}
+
+
+def _apply_explicit_form_controls( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply typed controls whose mapper omission represents a native clear."""
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ controls = {
+ **_COMMON_EXPLICIT_FORM_CONTROLS,
+ **_CHART_EXPLICIT_FORM_CONTROLS.get(config.chart_type, {}),
+ }
+ for field_name, (native_key, convert) in controls.items():
+ if field_name in explicit_fields:
+ converted = convert(getattr(config, field_name))
+ is_clear = (
+ converted is None
+ or converted is False
+ or converted == ""
+ or converted in ([], {})
+ )
+ if is_clear:
+ new_form_data[native_key] = converted
+
+ axis_controls = {
+ "xy": (
+ ("x_axis", "x_axis_title", "x_axis_format", None),
+ ("y_axis", "y_axis_title", "y_axis_format", "logAxis"),
+ ),
+ "mixed_timeseries": (
+ ("x_axis", "xAxisTitle", "x_axis_time_format", None),
+ ("y_axis", "yAxisTitle", "y_axis_format", "logAxis"),
+ (
+ "y_axis_secondary",
+ "yAxisTitleSecondary",
+ "y_axis_format_secondary",
+ "logAxisSecondary",
+ ),
+ ),
+ }
+ if config.chart_type in axis_controls:
+ for field_name, title_key, format_key, scale_key in axis_controls[
+ config.chart_type
+ ]:
+ if field_name not in explicit_fields:
+ continue
+ axis = getattr(config, field_name)
+ if axis is None:
+ new_form_data[title_key] = None
+ new_form_data[format_key] = None
+ if scale_key:
+ new_form_data[scale_key] = None
+ continue
+ axis_fields = set(axis.model_fields_set)
+ if "title" in axis_fields:
+ new_form_data[title_key] = axis.title
+ if "format" in axis_fields:
+ new_form_data[format_key] = axis.format
+ if scale_key and "scale" in axis_fields:
+ new_form_data[scale_key] = (
+ None if axis.scale is None else axis.scale == "log"
+ )
+
+ if config.chart_type == "xy" and "legend" in explicit_fields:
+ legend = config.legend
+ if legend is None:
+ new_form_data["show_legend"] = None
+ new_form_data["legendOrientation"] = None
+ else:
+ legend_fields = set(legend.model_fields_set)
+ if "show" in legend_fields:
+ new_form_data["show_legend"] = legend.show
+ if "position" in legend_fields:
+ new_form_data["legendOrientation"] = legend.position
+
+ if config.chart_type == "interactive_pivot":
+ if "temporal_column" in explicit_fields and config.temporal_column is None:
+ new_form_data["granularity_sqla"] = None
+ new_form_data["temporal_columns_lookup"] = None
+ if (
+ "series_limit_metric" in explicit_fields
+ and config.series_limit_metric is None
+ ):
+ new_form_data["series_limit_metric"] = None
+ if "comparison_period" in explicit_fields and config.comparison_period is None:
+ new_form_data["time_compare"] = None
+ if "comparison_type" in explicit_fields and config.comparison_type is None:
+ new_form_data["comparison_type"] = None
+
+ # A Waterfall axis replacement cannot inherit a bucket belonging to the old
+ # temporal subject. Grain omission preserves only while the axis is stable;
+ # explicit null is already handled by the declarative control map above.
+ if (
+ config.chart_type == "waterfall"
+ and existing_form_data.get("x_axis") != new_form_data.get("x_axis")
+ and "time_grain" not in explicit_fields
+ ):
+ new_form_data["time_grain_sqla"] = None
+
+
+def merge_same_viz_form_data(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Preserve saved controls that the typed mapper does not represent.
+
+ The typed MCP surface deliberately models a bounded subset of every Explore
+ control panel. For a replacement within the exact same native ``viz_type``,
+ keys absent from the mapper therefore represent omitted controls and retain
+ their saved values. Mapper output and the chart-specific merge helpers run
+ first and remain authoritative, including explicit empty, false, null, and
+ nested values.
+
+ No generic state crosses a visualization boundary. This prevents query-role
+ keys from the previous plugin (for example ``metric`` or ``groupby``) from
+ leaking into a different plugin whose role contract is unrelated.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if not isinstance(existing_viz_type, str) or existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+
+ _apply_explicit_form_controls(existing_form_data, new_form_data, config)
+
+ for key, value in existing_form_data.items():
+ if key in {"where", "having", "filters"}:
+ # merge_update_form_data already resolved these legacy predicates
+ # into adhoc_filters; restoring them would undo replacement/clear.
+ continue
+ if key == MCP_DASHBOARD_TIME_FILTER_SUBJECT:
+ # merge_update_form_data owns this provenance marker. Its absence
+ # may be an intentional subject clear and must not be undone by the
+ # generic preservation layer.
+ continue
+ if key not in new_form_data:
+ new_form_data[key] = value
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet controls without reinterpreting inherited query roles.
+
+ Saved Explore state may contain SQL dimensions and SQL WHERE/SIMPLE HAVING
+ filters beyond the typed authoring surface. Omitted roles are validated by
+ the native query contract and compilation, not as newly authored physical
+ columns or SIMPLE WHERE filters. Only this validation copy excludes them
+ and native query metadata; compiled and persisted form data stays intact.
+ Explicit replacements, including ``[]``, retain strict typed validation.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ for native_query_key in ("url_params", "extra_form_data", "extra_filters"):
+ validation_data.pop(native_query_key, None)
+ if update_config is None or isinstance(update_config, BulletChartConfig):
+ if update_config is None or update_config.dimensions is None:
+ validation_data.pop("groupby", None)
+ if update_config is None or "filters" not in update_config.model_fields_set:
+ validation_data.pop("adhoc_filters", None)
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ return BulletChartConfig.model_validate(validation_data)
Review Comment:
Confirmed: restating `dimensions` kept the inherited `orderby` in the strict validation copy, so a saved independent ranking sort failed as if newly authored. Fixed in e3dc3844f2e386b0f34b9a3596431c47c7a680ce: `validate_merged_bullet_form_data` also drops inherited `orderby`/`order_by_cols` unless the update supplies `order_by`, matching `groupby` and `adhoc_filters`; the native query contract still validates the inherited sort at compile. New `test_saved_bullet_restated_dimensions_keep_independent_sort` (saved `orderby: [["SavedRevenue", false]]`, update restates `dimensions: [{"name": "Region"}]`) fails before the fix with "Merged Bullet Chart configuration is invalid." and passes after.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,847 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
Review Comment:
Closed by 1a7455b21d4d6339beb650b6f093c4f7f55c5084 (same issue as the later thread on this line): `map_bullet_config` always writes `row_limit` (10000 when omitted), and the Bullet update merge restores the saved value when the update omits `row_limit`. Covered by `test_bullet_mapper_preserves_omission_and_honors_explicit_values` and `test_bullet_update_merge_row_limit_omission_and_explicit_value`.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2412,101 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ escape_separator(label)
Review Comment:
Confirmed: `query_context_processor` unescapes flattened column names, so the result key is `Revenue, net, EU` while the prefix was built from `Revenue\, net`. Fixed in 4211e3aad46e2fe8d8e528b348ed4eb875982ae2: field discovery matches the unescaped label (and still the escaped spelling from raw post-processing output), and the y-axis title uses the plain label; escaping stays only in the Vega fold field paths. New `test_xy_preview_renders_series_for_metric_label_with_separator` unescapes the real post-processed columns as the processor does and asserts both `Revenue, net, EU`/`Revenue, net, US` series fold; it fails without the fix.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -760,7 +1639,56 @@ def build_single_query_dict(
order_desc if order_desc is not None else form_data.get("order_desc", True)
)
qd["orderby"] = [(sort_metric, not descending)]
+ if orderby:
+ qd["orderby"] = orderby
+ for key in (
+ "annotation_layers",
+ "row_offset",
+ "series_columns",
+ "group_others_when_limit_reached",
+ "is_timeseries",
+ "time_offsets",
+ "time_compare_full_range",
+ ):
+ if key in form_data and form_data[key] is not None:
+ qd[key] = form_data[key]
+
+ # ``buildQueryObject`` keeps the modern series-limit controls, falls back
+ # to their legacy Timeseries names, and defaults the limit to zero. A
+ # malformed modern metric does not mask a valid legacy metric.
+ series_limit = form_data.get("series_limit")
+ if series_limit is None:
+ series_limit = form_data.get("limit")
+ if series_limit is not None:
+ qd["series_limit"] = series_limit
+ series_limit_metric = form_data.get("series_limit_metric")
+ if not _is_query_form_metric(series_limit_metric):
+ series_limit_metric = form_data.get("timeseries_limit_metric")
+ if series_limit_metric is not None:
+ qd["series_limit_metric"] = series_limit_metric
+ if apply_chart_fields is not None:
+ apply_chart_fields(qd, effective_row_limit)
apply_form_data_filters_to_query(qd, form_data)
+ # Mirror the common ``buildQueryObject``/``extractExtras`` translation used
+ # by native frontend plugins. ``granularity_sqla`` is a form-data control,
+ # while QueryObject calls the field ``granularity``; the SQL time grain is
+ # carried inside ``extras`` rather than as a top-level query field.
+ if include_common_temporal:
+ granularity = form_data.get("granularity") or form_data.get("granularity_sqla")
+ if granularity:
+ qd["granularity"] = granularity
+ if time_grain := form_data.get("time_grain_sqla"):
+ qd["extras"] = {
+ **(qd.get("extras") or {}),
+ "time_grain_sqla": time_grain,
Review Comment:
Confirmed: the MCP Table builder copied a cached `time_grain_sqla` into extras with no temporal column, and the semantic layer rejects it (`superset/semantic_layers/mapper.py`, "A time column must be specified when a time grain is provided."). Fixed in 951a56289db79cd1dbfbb6e788d1987a869e7eaf (plus the mypy narrowing in f92dbdfe0d6b169b9af9e69c6809515836ffb1f5): `build_table_query_dicts` ports `omitDormantGrain` for semantic-view aggregate mode, applied to every final query including totals/rowcount, with the same duration-grain set and the same `granularity`/`is_timeseries`/all-columns-`false`-in-`temporal_columns_lookup` conditions. New `test_semantic_table_aggregate_omits_dormant_time_grain` asserts the grain is dropped for `semantic_view` and kept for `table` and for a temporal groupby; it fails without the fix.
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1408,6 +1421,129 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Inherited native SQL dimensions remain query expressions when updating metrics
+or `order_by`; reference their output labels to sort by them. Caller-supplied
+`dimensions` must still be physical columns, not SQL expressions.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling. Bullet metric strings use
+JavaScript numeric spellings: underscore separators and non-ASCII digits are
+rejected rather than interpreted as numbers. Bounded array-valued dimensions
+retain their raw values in data reads and exports; preview category labels use
+JavaScript string conversion (for example, `[1, 2]` displays as `1,2`). Native
+SQL metrics without a label use their SQL expression as the output label.
+
+Presentation-only updates preserve saved predicates, including legacy top-level
+`where`, `having`, and `filters`, and normalize the native `order_by_cols` alias.
+When both `orderby` and `order_by_cols` are saved, their sort entries are
+concatenated in form-data key order, including when `orderby` is empty.
+Explicit `filters: []` and `order_by: []` clear those inherited controls.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+Labels that become empty after sanitization retain their slots so later labels
+remain aligned with their corresponding values.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates. Omitted label controls preserve the saved state only when their
+corresponding value controls (`ranges`, `markers`, or `marker_lines`) are also
+omitted. Replacing a value control without its labels clears the saved labels;
+resend the labels to retain annotations with the replacement values.
+
+Saved native `ranges`, `markers`, and `marker_lines` controls ignore empty,
+non-numeric, and NaN tokens, matching Explore. If no numeric range remains, the
+preview uses the default band up to 110% of the largest measure. Unrelated
+updates preserve these saved controls. Newly authored typed lists must contain
+finite numbers; infinite native values and oversized controls remain errors.
+
+On same-dataset chart updates, `filters: []` clears both user filters and the
+generated dashboard-time binding. `temporal_column: null` clears only that
+generated binding, preserving user filters. Omitting those controls preserves
+the saved binding. Bullet Vega previews keep separate indexed rows even when
+dimension display labels are identical, and support both `SMART_NUMBER` and
+`SMART_NUMBER_SIGNED` number formats. Bullet preview numeric format
+precision is limited to 20 digits before formatting; raw data reads do not
+validate presentation formats.
+
+Native Bullet temporal filters retain the active filter's subject and range as a
+pair; `No filter` placeholders do not supply the subject of another active range.
+Typed configs support one such pair. Multiple active native temporal filters, or
+conflicts with explicit `temporal_column`/`time_range`, are rejected rather than
+silently dropping or moving a predicate.
+
+### Query result limits
+
+MCP chart tools accept up to **50,000 rows per query** and **100,000 total rows**
+across all queries in one result. The aggregate counts each returned data row,
+including repeated rows in different queries; it does not use `rowcount` or
+`total_rows` metadata. The row-shaped `indexnames`
+array emitted by Chart Data uses the same 50,000-entry limit, rather than the
+4,096-entry limit for other metadata arrays. Index entries count toward the
+shared row-data work budget and retain the 1 MiB aggregate metadata byte limit.
+Multi-dimension pivot index tuples are serialized as JSON arrays. Nested
+containers within an index entry retain the standard container limits.
+
+The shared query-result validator also enforces **non-configurable hard limits**:
+
+- **2,500,000 values/containers across all queries**. Each row object, cell
+ scalar (including null), and nested list, tuple, array, or object contributes
+ to this shared work budget; nested elements count individually and repeated
+ occurrences count again. Object keys count toward byte limits, not this value
+ count. Row-shaped `indexnames` and their entries also consume this budget.
+ For example, 50,000 rows with 50 scalar columns require 2,550,000 values
+ (50,000 row objects plus 2,500,000 cells) and exceed the budget even if their
+ encoded size is below 16 MiB.
+- **64 KiB (65,536 UTF-8 bytes) per row-cell string value**. A single
+ 65,537-byte cell fails even in a one-row result. Binary cells must also fit
+ the cell limit before and after conversion to UTF-8 or a `base64:` string.
+- **16 MiB (16,777,216 bytes) of aggregate JSON-encoded result data and
+ metadata** across all queries, including escaped strings, keys, and container
+ syntax. Metadata strings (including SQL text) are not subject to the row-cell
+ string cap; all metadata shares a separate 1 MiB aggregate allowance.
+
+These limits apply before rendering or exporting in `get_chart_data`,
+`get_chart_preview`, chart generation/update compile checks, `query_dataset`,
+and semantic-layer `get_table`. They include inline/table responses and MCP
+CSV, Excel, and Parquet export paths; exports do not bypass source validation.
Review Comment:
Agreed: `GetChartDataRequest.format` is `Literal["json", "csv", "excel"]` and no MCP tool exports Parquet. Fixed in 49dee0e733ca17d06cdfd18fa614489266d67514: both `docs/admin_docs/configuration/mcp-server.mdx` and `UPDATING.md` now say the `get_chart_data` CSV and Excel export formats.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1738,564 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ # Native extractQueryFields excludes empty-string column references.
+ table_columns = [
+ column
+ for column in _as_list(
+ form_data.get("all_columns") if raw_mode else form_data.get("groupby")
+ )
+ if column != ""
+ ]
+ table_metrics = [] if raw_mode else _as_list(form_data.get("metrics"))
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ query_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ query_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ # Native comparisons use ordinary metrics, before percentage-only metrics
+ # are added to the selected query and contribution operator.
+ has_time_comparison = _time_comparison(form_data, table_metrics)
+ offsets = _table_time_offsets(form_data, {**query, "metrics": table_metrics})
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if has_time_comparison:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if has_time_comparison and offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ # ``query["row_limit"]`` is the normalized caller limit (explicit request
+ # limit or the saved row_limit, which may be stored as a string); page
+ # sizing narrows it but never replaces it.
+ configured_limit = _positive_int(query.get("row_limit"))
+ if form_data.get("server_pagination"):
+ if page_size := _positive_int(form_data.get("server_page_length")):
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if query_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ query_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
+ if metric is None:
+ plural_metrics = _as_list(form_data.get("metrics"))
+ metric = plural_metrics[0] if plural_metrics else None
+ columns = _resolve_big_number_query_columns(form_data) if trendline else []
+ query = build_single_query_dict(
+ form_data,
+ columns,
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if trendline:
+ # Big Number has no series dimension. Its frontend pivot receives
+ # the common base QueryObject (whose columns are empty), not the
+ # final QueryObject after the explicit x-axis is added. Preserve
+ # that distinction instead of falling back to the final columns.
+ query["series_columns"] = []
+ if not form_data.get("x_axis"):
+ query["is_timeseries"] = True
+ query["post_processing"] = _timeseries_post_processing(form_data, query)
+ if form_data.get("aggregation") == "raw":
+ return [
+ query,
+ {
+ **query,
+ "columns": [],
+ "is_timeseries": False,
+ "post_processing": [],
+ },
+ ]
+ return [query]
+
+
+def build_waterfall_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Waterfall buildQuery with raw-axis ordering."""
+ metrics, groupby = resolve_metrics_and_groupby(form_data)
+ # normalizeTimeColumn runs after Waterfall's buildQuery callback. It
+ # wraps only the final x-axis column; orderby deliberately retains the
+ # raw control value produced inside the callback.
+ raw_axis = form_data.get("x_axis") or form_data.get("granularity_sqla")
+ query_axis = (
+ _normalized_x_axis_query_field(form_data)
+ if form_data.get("x_axis")
+ else raw_axis
+ )
+ waterfall_columns = ([query_axis] if query_axis else []) + groupby
+ raw_ordering_columns = ([raw_axis] if raw_axis else []) + groupby
+ query = build_single_query_dict(
+ form_data,
+ waterfall_columns,
+ metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=None,
+ )
+ query["orderby"] = [[column, True] for column in raw_ordering_columns]
+ if form_data.get("x_axis"):
+ query.pop("is_timeseries", None)
+ return [query]
+
+
+def build_mixed_timeseries_query_dicts( # noqa: C901
Review Comment:
Agreed: neither `build_mixed_timeseries_secondary` nor `with_x_axis_column` has a caller in `superset/` or `tests/`. Removed in 9989731953eebcbb9ea154db115ebcadfa74e22c; their remaining helpers (`build_single_query_dict`, `extract_x_axis_col`, `split_adhoc_filters_into_base_filters`) still have other callers, and ruff/mypy pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4236685382
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1809,7 +1815,1047 @@ def generate_xy_vega_lite_preview(
spec["transform"] = [
{"fold": series_columns, "as": [series_field, value_field]}
]
- encoding["y"] = {"field": value_field, "type": "quantitative"}
+ # Vega-Lite stacks bar and area marks colored by a nominal field
+ # by default; follow the chart's saved ``stack`` control instead.
+ encoding["y"] = {
Review Comment:
With `stack: None` and no per-series offset, the folded series are drawn on top of each other. For an unstacked `echarts_timeseries_bar` with `groupby=["region"]` and one row holding `"revenue, East": 10` and `"revenue, West": 20`, both bars sit at the same x and the 20 hides the 10, whereas Explore shows them side by side. Should the unstacked bar case add an `xOffset` on the series field (or otherwise group the bars) instead of only clearing `stack`?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,568 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from copy import deepcopy
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ bullet_groupby_list,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ TablePreview,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Validate Bullet result rows without requiring a backend preview formatter."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data), validate_format=False)
+ except BulletOutputError as ex:
+ return ChartError(error=bounded_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+def _unsupported_table_preview() -> ChartError:
+ return ChartError(
+ error=(
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ ),
+ error_type="UnsupportedFormat",
+ )
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # A null ``data`` array is a malformed producer result, not an empty chart.
+ null_data_is_empty = False
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ validates_native_references = True
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ # A merged validation copy omits native dimensions; its native sorts
+ # belong to query-contract validation, not physical-column resolution.
+ orders = (
+ config_dict.get("order_by") or []
+ if config.dimensions is not None or config._inherited_groupby is not None
+ else []
+ )
+ for order in orders:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.order_dimensions, config.metric
+ )
+ if role == "dimension" and index is not None:
+ if config.dimensions is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized._inherited_groupby = deepcopy(config._inherited_groupby)
+ normalized.validate_roles_and_outputs()
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def build_query_dicts(
+ self,
+ form_data: dict[str, Any],
+ *,
+ viz_type: str,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+ ) -> list[dict[str, Any]] | None:
+ """Mirror ``Bullet/buildQuery``: one metric over the groupby hierarchy.
+
+ The frontend keeps only explicitly saved ordering; it never applies
+ the legacy top-N metric ordering of unadapted visualizations.
+ """
+ from superset.common.form_data_query_context import (
+ build_query_objects_from_form_data,
+ )
+
+ queries = build_query_objects_from_form_data(
+ form_data,
+ viz_type=viz_type,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ filters_prepared=True,
+ )
+ query = queries[0]
+ metric = form_data.get("metric")
+ query["columns"] = bullet_groupby_list(form_data.get("groupby"))
+ query["metrics"] = [metric] if metric else []
+ if not form_data.get("orderby") and not form_data.get("order_by_cols"):
+ query["orderby"] = []
+ return queries
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import first_query_data
+
+ rows, failure = first_query_data(
+ result, none_as_empty=False, temporal_json_numbers=True
+ )
+ if failure is not None:
+ return failure
+ assert rows is not None
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose validated rows independently of backend preview formatter support."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ model = resolve_bullet_render_model(
+ data, dict(form_data), validate_format=False
+ )
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+ if not data:
+ # The strict model's zero-valued ungrouped row is a render-only
+ # frontend fallback, not source query data, so an empty query
+ # exposes no rows to get-data and exports.
+ return [], None
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the dimensions. Its float measure is
+ # render-only: exported rows keep the validated source metric so exact
+ # BIGINT/Decimal values are not rounded to binary64.
+ rows: list[Any] = []
+ for source, row in zip(data, model.rows, strict=False):
Review Comment:
`model.rows` is cut to the first row for an ungrouped Bullet, and this `zip` stops there, so `get_chart_data` JSON/CSV/Excel exports drop every later source row. For an ungrouped saved metric that yields several rows (e.g. `SUM(revenue) OVER ()`), the export silently contains one. The existing multi-row preservation tests use grouped charts and the ungrouped export test only covers an empty result. Should the exposed rows come from the validated source data rather than the render-limited model, with a test returning two ungrouped rows through `get_chart_data` and asserting both survive?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,568 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from copy import deepcopy
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ bullet_groupby_list,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ TablePreview,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Validate Bullet result rows without requiring a backend preview formatter."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data), validate_format=False)
+ except BulletOutputError as ex:
+ return ChartError(error=bounded_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+def _unsupported_table_preview() -> ChartError:
+ return ChartError(
+ error=(
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ ),
+ error_type="UnsupportedFormat",
+ )
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # A null ``data`` array is a malformed producer result, not an empty chart.
+ null_data_is_empty = False
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ validates_native_references = True
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ # A merged validation copy omits native dimensions; its native sorts
+ # belong to query-contract validation, not physical-column resolution.
+ orders = (
+ config_dict.get("order_by") or []
+ if config.dimensions is not None or config._inherited_groupby is not None
+ else []
+ )
+ for order in orders:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.order_dimensions, config.metric
+ )
+ if role == "dimension" and index is not None:
+ if config.dimensions is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized._inherited_groupby = deepcopy(config._inherited_groupby)
+ normalized.validate_roles_and_outputs()
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def build_query_dicts(
+ self,
+ form_data: dict[str, Any],
+ *,
+ viz_type: str,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+ ) -> list[dict[str, Any]] | None:
+ """Mirror ``Bullet/buildQuery``: one metric over the groupby hierarchy.
+
+ The frontend keeps only explicitly saved ordering; it never applies
+ the legacy top-N metric ordering of unadapted visualizations.
+ """
+ from superset.common.form_data_query_context import (
+ build_query_objects_from_form_data,
+ )
+
+ queries = build_query_objects_from_form_data(
+ form_data,
+ viz_type=viz_type,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ filters_prepared=True,
+ )
+ query = queries[0]
+ metric = form_data.get("metric")
+ query["columns"] = bullet_groupby_list(form_data.get("groupby"))
+ query["metrics"] = [metric] if metric else []
+ if not form_data.get("orderby") and not form_data.get("order_by_cols"):
+ query["orderby"] = []
+ return queries
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import first_query_data
+
+ rows, failure = first_query_data(
+ result, none_as_empty=False, temporal_json_numbers=True
+ )
+ if failure is not None:
+ return failure
+ assert rows is not None
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose validated rows independently of backend preview formatter support."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ model = resolve_bullet_render_model(
+ data, dict(form_data), validate_format=False
+ )
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+ if not data:
+ # The strict model's zero-valued ungrouped row is a render-only
+ # frontend fallback, not source query data, so an empty query
+ # exposes no rows to get-data and exports.
+ return [], None
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the dimensions. Its float measure is
+ # render-only: exported rows keep the validated source metric so exact
+ # BIGINT/Decimal values are not rounded to binary64.
+ rows: list[Any] = []
+ for source, row in zip(data, model.rows, strict=False):
+ exposed = dict(row)
+ if model.metric_field in source:
+ exposed[model.metric_field] = _safe_enum_backing(
+ source[model.metric_field]
+ )
+ rows.append(exposed)
+ return rows, None
+
+ def ascii_preview(
+ self,
+ data: list[Any],
+ form_data: dict[str, Any],
+ width: int,
+ height: int = 20,
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+
+ def table_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> TablePreview | ChartError | None:
+ return _unsupported_table_preview()
+
+ def unsupported_preview(self, preview_format: str) -> ChartError | None:
+ if preview_format == "table":
+ return _unsupported_table_preview()
+ return None
+
+ def vega_lite_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> VegaLitePreview | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_bullet_vega_lite_preview,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ return _generate_bullet_vega_lite_preview(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+
+ def resolve_update_config(
+ self,
+ config: Any,
+ existing_form_data: dict[str, Any],
+ *,
+ dataset_rebind: bool,
+ ) -> Any:
+ """Resolve sort targets against the saved hierarchy before mapping."""
+ if not isinstance(config, BulletChartConfig) or config.dimensions is not None:
+ return config
+ if not config.order_by:
+ return config
+ dimensions = (
+ bullet_groupby_list(existing_form_data.get("groupby"))
+ if not dataset_rebind and existing_form_data.get("viz_type") == "bullet"
+ else []
+ )
+ resolved = config.model_copy(deep=True)
+ resolved._inherited_groupby = deepcopy(dimensions)
+ return resolved.validate_roles_and_outputs()
+
+ def merge_update_form_data(
+ self,
+ existing_form_data: dict[str, Any],
+ new_form_data: dict[str, Any],
+ config: Any,
+ *,
+ dataset_rebind: bool,
+ ) -> dict[str, Any] | None:
+ """Merge filter provenance and preserve omitted native Bullet controls."""
+ if not isinstance(config, BulletChartConfig) or dataset_rebind:
+ return None
+ from superset.mcp_service.chart.chart_utils import (
+ _normalize_bullet_query_aliases,
+ merge_bullet_form_data,
+ merge_update_form_data,
+ )
+
+ saved_form_data = existing_form_data
+ if "orderby" in new_form_data:
+ # An explicit ordering replaces every saved ordering alias, so a
+ # stale or malformed saved alias is never parsed or inherited.
+ existing_form_data = {
+ key: value
+ for key, value in existing_form_data.items()
+ if key not in {"orderby", "order_by_cols"}
+ }
+ if existing_form_data.get("viz_type") == "bullet":
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ merged = dict(new_form_data)
+ if "row_limit" not in config.model_fields_set:
+ # The mapper always emits the schema default; an omitted limit
+ # keeps the saved value instead.
+ merged.pop("row_limit", None)
+ merge_update_form_data(existing_form_data, merged, config)
+ merge_bullet_form_data(existing_form_data, merged)
+ if "row_limit" not in merged and "row_limit" in new_form_data:
+ merged["row_limit"] = new_form_data["row_limit"]
+ # Like the shared overlay, unmodeled native presentation controls
+ # survive a same-viz update; query roles of other visualizations and
+ # legacy predicate aliases (already folded into adhoc filters) do not.
+ for key, value in saved_form_data.items():
+ if (
+ key not in merged
+ and key not in self.query_role_keys
+ and key not in {"where", "having", "filters"}
+ ):
+ merged[key] = value
Review Comment:
This loop re-adds the saved `_mcp_dashboard_time_filter_subject` after an explicit filter clear. For a saved Bullet that has a dashboard-time binding, `update_chart` with `filters: []` makes the shared merge and `merge_bullet_form_data` drop both `adhoc_filters` and the marker, but the marker is neither in `query_role_keys` nor in the `where/having/filters` exclusion, so it is copied back here. Merged-state validation then fails with `must match exactly one TEMPORAL_RANGE filter ... found 0`, so a plain "remove this chart's filters" update is rejected. Should the provenance marker be excluded from this restoration when `adhoc_filters` is in `merged`?
##########
superset/mcp_service/chart/compile.py:
##########
@@ -451,7 +456,474 @@ def _build_compile_error(message: str) -> ChartGenerationError:
)
-def validate_and_compile(
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ # Match QueryObject's guarded legacy saved-metric normalization.
+ if not ({"sqlExpression", "aggregate", "column"} & value.keys()):
+ label = value.get("label")
+ if isinstance(label, str) and label:
+ return "saved_metric", label
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", bounded_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
Review Comment:
Is this Timeseries/Mixed/Deck branch reachable? The only caller of `_native_reference_error` (in `validate_and_compile`) is gated by `state_plugin.validates_native_references`, and Bullet is the only plugin that sets it, so `viz_type` is always `bullet` here. For a config-less same-dataset rebind of a Timeseries chart with a stale ranking metric, these checks never run, and the direct-helper tests that call this function bypass the gate. Should the branches be removed, or the owning plugins opted in so they actually run?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4235975815
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -3383,7 +3414,620 @@ def _as_column_list(value: Any) -> list[Any]:
return [value]
-def analyze_chart_capabilities(viz_type: str | None, config: Any) -> ChartCapabilities:
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ # A missing saved viz_type is a boundary too, matching merge_chart_form_data.
+ if existing_form_data.get("viz_type") != new_form_data.get("viz_type"):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
Review Comment:
Good catch, confirmed with a failing test and fixed in bcf24ce6ed21d430dd1d207ddc9671071132282d.
When `temporal_column` is omitted, `filters` is omitted, `time_range` is supplied and the chart has no MCP subject marker, `merge_update_form_data` now adopts the saved filter as the binding if exactly one native SIMPLE/WHERE `TEMPORAL_RANGE` filter exists, whatever its column. The update rewrites it in place and keeps its subject. Your example now yields a single `ShipDate` filter with "Last month", and `time_range: null` resets that same filter to "No filter" (no second binding). This matches master's shared Gantt reconciliation, which also treats exactly one native SIMPLE/WHERE TEMPORAL_RANGE as the binding for an explicit temporal update.
With zero such filters the behaviour is unchanged: the dataset-default binding is added. With several, none is adopted because picking one would be a guess. The saved filters stay byte-for-byte and the default binding is appended. An explicit `temporal_column` (or a changed native subject) still takes precedence as before.
Tests (`tests/unit_tests/mcp_service/chart/test_bullet_chart.py`):
- `test_bullet_range_only_update_adopts_single_unmarked_native_binding`: update and null-clear across the immediate, preview-first and cached update paths.
- `test_bullet_range_only_update_keeps_several_unmarked_native_bindings`: pins the ambiguous case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4239936655
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -3489,7 +3520,643 @@ def _as_column_list(value: Any) -> list[Any]:
return [value]
-def analyze_chart_capabilities(viz_type: str | None, config: Any) -> ChartCapabilities:
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ # A missing saved viz_type is a boundary too, matching merge_chart_form_data.
+ if existing_form_data.get("viz_type") != new_form_data.get("viz_type"):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+ if (
+ existing_binding is None
+ and existing_subject is None
+ and range_explicit
+ and not subject_authoritative
+ and "filters" not in explicit_fields
+ ):
+ # A range-only update on an unmarked chart adopts its saved temporal
+ # filter even when Explore bound it to a column other than the mapper's
+ # proposed subject. Zero or several native ranges stay ambiguous, so the
+ # proposed binding is added instead of guessing which one to rewrite.
+ unmarked_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("expressionType") == "SIMPLE"
+ and filter_.get("clause") == "WHERE"
+ and isinstance(filter_.get("subject"), str)
+ ]
+ if len(unmarked_matches) == 1:
+ existing_binding = unmarked_matches[0]
+ existing_subject = existing_binding["subject"]
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet controls without reinterpreting inherited query roles.
+
+ Saved Explore state may contain SQL dimensions and SQL WHERE/SIMPLE HAVING
+ filters beyond the typed authoring surface. Omitted roles are validated by
+ the native query contract and compilation, not as newly authored physical
+ columns or SIMPLE WHERE filters. Only this validation copy excludes them
+ and native query metadata; compiled and persisted form data stays intact.
+ Explicit replacements, including ``[]``, retain strict typed validation.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ for native_query_key in ("url_params", "extra_form_data", "extra_filters"):
+ validation_data.pop(native_query_key, None)
+ if not validation_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT):
+ # A null marker records an explicit subject clear with no binding to
+ # validate; it is not an authored physical column.
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ if update_config is None or isinstance(update_config, BulletChartConfig):
+ if update_config is None or update_config.dimensions is None:
+ validation_data.pop("groupby", None)
+ if update_config is None or "filters" not in update_config.model_fields_set:
+ validation_data.pop("adhoc_filters", None)
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ if update_config is None or "order_by" not in update_config.model_fields_set:
+ # An inherited sort may rank by an independent saved metric; the
+ # native query contract validates it, not the authoring schema.
+ for order_key in ("orderby", "order_by_cols"):
+ validation_data.pop(order_key, None)
+ return BulletChartConfig.model_validate(validation_data)
Review Comment:
This validates the whole merged form_data, but the shared preservation registry has already copied keys such as `color_scheme`, `number_format`, and `legendOrientation` into it, and `BulletChartConfig` rejects those as unknown fields. Converting a saved Pie chart that has `color_scheme` set with `{"chart_type":"bullet","metric":{"name":"revenue","aggregate":"SUM"}}` would then return `Unknown field 'color_scheme'` instead of updating the chart. Should the validation copy drop preserved presentation/envelope keys that the typed Bullet schema doesn't model, the way it already drops `url_params` and `extra_form_data`?
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -902,6 +912,9 @@ async def _get_chart_data( # noqa: C901
if result_error := validate_query_result_envelope(
result,
none_as_empty=null_data_is_empty(effective_form_data.get("viz_type")),
+ temporal_json_numbers=temporal_json_numbers(
Review Comment:
Bullet turns this on, so temporal cells reach the exporters as epoch-millisecond numbers, but `_excel_temporal_value` only restores dates from ISO strings. A saved Bullet grouped by `order_date` fetched with `format: "excel"` would produce a workbook whose date column holds values like `1704153600000` instead of Excel dates. Should the Excel path rebuild dates from the epoch values for temporal columns, or export from the ISO-text projection?
##########
superset/mcp_service/chart/plugin.py:
##########
@@ -315,6 +318,24 @@ def schema_error_hint(self) -> ChartGenerationError | None:
invalid_result_message: ClassVar[str]
invalid_result_suggestions: ClassVar[tuple[str, ...]]
+ #: Whether chart-data consumers convert temporal and duration result
+ #: values to the JSON numbers and text the frontend ``transformProps``
+ #: receives, instead of the generic scalar serialization.
+ temporal_json_numbers: ClassVar[bool]
+
+ #: Whether raw get_chart_data rows and CSV/XLSX exports keep exact
+ #: non-finite floats (JSON responses still sanitize them to null).
+ preserve_nonfinite_floats: ClassVar[bool]
Review Comment:
`preserve_nonfinite_floats` is declared here and defaulted on the base class, but nothing reads it, and normalization still replaces non-finite floats with `None` unconditionally. A plugin that sets it to `True` would still get null/blank values in raw rows and CSV/XLSX exports. Should this flag be wired into normalization/export or removed until something needs it?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,589 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from copy import deepcopy
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ bullet_groupby_list,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ resolve_bullet_order_target,
+ TablePreview,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Validate Bullet result rows without requiring a backend preview formatter."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data), validate_format=False)
+ except BulletOutputError as ex:
+ return ChartError(error=bounded_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+def _unsupported_table_preview() -> ChartError:
+ return ChartError(
+ error=(
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ ),
+ error_type="UnsupportedFormat",
+ )
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # A null ``data`` array is a malformed producer result, not an empty chart.
+ null_data_is_empty = False
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ validates_native_references = True
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ # A merged validation copy omits native dimensions; its native sorts
+ # belong to query-contract validation, not physical-column resolution.
+ orders = (
+ config_dict.get("order_by") or []
+ if config.dimensions is not None or config._inherited_groupby is not None
+ else []
+ )
+ for order in orders:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.order_dimensions, config.metric
+ )
+ if role == "dimension" and index is not None:
+ if config.dimensions is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized._inherited_groupby = deepcopy(config._inherited_groupby)
+ normalized.validate_roles_and_outputs()
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def build_query_dicts(
+ self,
+ form_data: dict[str, Any],
+ *,
+ viz_type: str,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+ ) -> list[dict[str, Any]] | None:
+ """Mirror ``Bullet/buildQuery``: one metric over the groupby hierarchy.
+
+ The frontend keeps only explicitly saved ordering; it never applies
+ the legacy top-N metric ordering of unadapted visualizations.
+ """
+ from superset.common.form_data_query_context import (
+ build_query_objects_from_form_data,
+ )
+
+ queries = build_query_objects_from_form_data(
+ form_data,
+ viz_type=viz_type,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ filters_prepared=True,
+ )
+ query = queries[0]
+ metric = form_data.get("metric")
+ query["columns"] = bullet_groupby_list(form_data.get("groupby"))
+ query["metrics"] = [metric] if metric else []
+ if not form_data.get("orderby") and not form_data.get("order_by_cols"):
+ query["orderby"] = []
+ return queries
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import first_query_data
+
+ rows, failure = first_query_data(
+ result, none_as_empty=False, temporal_json_numbers=True
+ )
+ if failure is not None:
+ return failure
+ assert rows is not None
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose validated rows independently of backend preview formatter support."""
+ from superset.mcp_service.chart.preview_utils import (
+ _safe_enum_backing,
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ model = resolve_bullet_render_model(
+ data, dict(form_data), validate_format=False
+ )
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+ if not data:
+ # The strict model's zero-valued ungrouped row is a render-only
+ # frontend fallback, not source query data, so an empty query
+ # exposes no rows to get-data and exports.
+ return [], None
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the dimensions. Its float measure is
+ # render-only: exported rows keep the validated source metric so exact
+ # BIGINT/Decimal values are not rounded to binary64.
+ # An ungrouped model keeps only the rendered first row; every later
+ # source row was validated by the same resolver and stays exported.
+ model_rows = list(model.rows)
+ if not model.dimensions:
+ try:
+ for source in data[len(model_rows) :]:
+ model_rows.extend(
+ resolve_bullet_render_model(
+ [source], dict(form_data), validate_format=False
+ ).rows
+ )
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=bounded_exception_message(ex), error_type=ex.error_type
+ )
+ rows: list[Any] = []
+ for source, row in zip(data, model_rows, strict=True):
+ exposed = dict(row)
+ if model.metric_field in source:
+ exposed[model.metric_field] = _safe_enum_backing(
+ source[model.metric_field]
+ )
+ rows.append(exposed)
+ return rows, None
+
+ def ascii_preview(
+ self,
+ data: list[Any],
+ form_data: dict[str, Any],
+ width: int,
+ height: int = 20,
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.response_preflight import (
+ bounded_exception_message,
+ )
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
Review Comment:
`width` and `height` are dropped here, and the saved-chart preview path in `get_chart_preview.py` returns this text without the clipping that `_generate_ascii_preview_from_data` applies. A saved Bullet preview requested with `ascii_width=40, ascii_height=5` would return the fixed-width separator and every line, while reporting a 40x5 frame. Should the saved path apply the same width/height truncation as the unsaved path?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5677021964
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5800377517
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4172336856
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the selected roles. Its zero-valued
+ # ungrouped row is a render-only frontend fallback, not source query
+ # data, so an empty query exposes no rows to get-data and exports.
+ return (model.rows if data else []), None
+
+ def ascii_preview(
+ self, data: list[Any], form_data: dict[str, Any], width: int
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def vega_lite_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> VegaLitePreview | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_bullet_vega_lite_preview,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_bullet_vega_lite_preview(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def resolve_update_config(
+ self,
+ config: Any,
+ existing_form_data: dict[str, Any],
+ *,
+ dataset_rebind: bool,
+ ) -> Any:
+ """Resolve sort targets against the saved hierarchy before mapping."""
+ if not isinstance(config, BulletChartConfig) or config.dimensions is not None:
+ return config
+ if not config.order_by:
+ return config
+ dimensions = (
+ existing_form_data.get("groupby") or []
+ if not dataset_rebind and existing_form_data.get("viz_type") == "bullet"
+ else []
+ )
+ return BulletChartConfig.model_validate(
Review Comment:
fixed in df03f8e50b: saved updates prune the saved roles against the replacement dataset and resolve the omitted hierarchy, matching the cached-preview path.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1750,523 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
+ (form_data.get("all_columns") or [])
+ if raw_mode
+ else (form_data.get("groupby") or [])
+ )
+ table_metrics = [] if raw_mode else list(form_data.get("metrics") or [])
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ query["columns"] = [
+ _temporal_column(column, form_data) for column in table_columns
+ ]
+ offsets = _table_time_offsets(form_data, query)
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if offsets:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ configured_limit = form_data.get("row_limit")
+ if form_data.get("server_pagination"):
+ page_size = form_data.get("server_page_length") or 0
+ if page_size:
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if table_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ table_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
+ columns = _resolve_big_number_query_columns(form_data) if trendline else []
+ query = build_single_query_dict(
+ form_data,
+ columns,
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if trendline:
+ # Big Number has no series dimension. Its frontend pivot receives
+ # the common base QueryObject (whose columns are empty), not the
+ # final QueryObject after the explicit x-axis is added. Preserve
+ # that distinction instead of falling back to the final columns.
+ query["series_columns"] = []
+ if not form_data.get("x_axis"):
+ query["is_timeseries"] = True
+ query["post_processing"] = _timeseries_post_processing(form_data, query)
+ if form_data.get("aggregation") == "raw":
+ return [
+ query,
+ {
+ **query,
+ "columns": [],
+ "is_timeseries": False,
+ "post_processing": [],
+ },
+ ]
+ return [query]
+
+
+def build_waterfall_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Waterfall buildQuery with raw-axis ordering."""
+ metrics, groupby = resolve_metrics_and_groupby(form_data)
+ # normalizeTimeColumn runs after Waterfall's buildQuery callback. It
+ # wraps only the final x-axis column; orderby deliberately retains the
+ # raw control value produced inside the callback.
+ raw_axis = form_data.get("x_axis") or form_data.get("granularity_sqla")
+ query_axis = (
+ _normalized_x_axis_query_field(form_data)
+ if form_data.get("x_axis")
+ else raw_axis
+ )
+ waterfall_columns = ([query_axis] if query_axis else []) + groupby
+ raw_ordering_columns = ([raw_axis] if raw_axis else []) + groupby
+ query = build_single_query_dict(
+ form_data,
+ waterfall_columns,
+ metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=None,
+ )
+ query["orderby"] = [[column, True] for column in raw_ordering_columns]
+ if form_data.get("x_axis"):
+ query.pop("is_timeseries", None)
+ return [query]
+
+
+def build_mixed_timeseries_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Mixed Timeseries buildQuery for both independent layers."""
+ from superset.utils.core import split_adhoc_filters_into_base_filters
+
+ queries: list[dict[str, Any]] = []
+ x_axis = _normalized_x_axis_query_field(form_data)
+ for secondary in (False, True):
+ layer = _mixed_layer_form_data(form_data, secondary=secondary)
+ if secondary and form_data.get("adhoc_filters_b") is not None:
+ for key in ("filters", "where", "having"):
+ layer.pop(key, None)
+ layer["adhoc_filters"] = form_data.get("adhoc_filters_b") or []
Review Comment:
fixed in df03f8e50b: request-level (isExtra) adhoc filters are now applied to the secondary layer on top of adhoc_filters_b.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1781,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+ """
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
Review Comment:
fixed in df03f8e50b: a time_range update now replaces the single unmarked native binding for that subject in place and records provenance.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1781,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
Review Comment:
fixed in df03f8e50b: sort entries on dimensions removed from groupby are dropped; metric sorts and still-grouped dimension sorts are kept.
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the selected roles. Its zero-valued
+ # ungrouped row is a render-only frontend fallback, not source query
+ # data, so an empty query exposes no rows to get-data and exports.
+ return (model.rows if data else []), None
Review Comment:
fixed in df03f8e50b: get-data and exports now keep the validated source metric; the float is only used for rendering.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1750,523 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
+ (form_data.get("all_columns") or [])
+ if raw_mode
+ else (form_data.get("groupby") or [])
+ )
+ table_metrics = [] if raw_mode else list(form_data.get("metrics") or [])
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ query["columns"] = [
+ _temporal_column(column, form_data) for column in table_columns
+ ]
+ offsets = _table_time_offsets(form_data, query)
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if offsets:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ configured_limit = form_data.get("row_limit")
+ if form_data.get("server_pagination"):
+ page_size = form_data.get("server_page_length") or 0
+ if page_size:
Review Comment:
fixed in df03f8e50b: the normalized limit is coerced to an int and only narrowed by the UI page size; CSV/Excel downloads skip page sizing.
##########
tests/unit_tests/mcp_service/chart/tool/test_get_chart_data.py:
##########
@@ -2530,106 +2990,2240 @@ def test_multi_query_row_count_reflects_limit() -> None:
]
-@pytest.mark.asyncio
-async def test_unsaved_mixed_timeseries_returns_nonempty_secondary_query(
+def _multi_query_chart_data(
+ query_results: list[dict[str, Any]],
+) -> ChartData:
+ nested_results = _build_query_results(query_results, None)
+ assert nested_results is not None
+ first_data = query_results[0]["data"]
+ return ChartData(
+ chart_id=1,
+ chart_name="Multi-query",
+ chart_type="mixed_timeseries",
+ columns=[],
+ data=first_data,
+ query_results=nested_results,
+ row_count=len(first_data),
+ total_rows=len(first_data),
+ summary="Multi-query result",
+ insights=[],
+ data_quality={},
+ recommended_visualizations=[],
+ data_freshness=None,
+ performance=PerformanceMetadata(query_duration_ms=1, cache_status="fresh"),
+ )
+
+
+def test_complete_response_budget_charges_multi_query_first_leg_alias(
monkeypatch: pytest.MonkeyPatch,
) -> None:
- """The tool response must not collapse a multi-query command result."""
- from unittest.mock import AsyncMock
-
- get_data_command_module = importlib.import_module(
- "superset.commands.chart.data.get_data_command"
- )
- chart_data_module = importlib.import_module(
- "superset.mcp_service.chart.tool.get_chart_data"
+ row = {f"c{index}": 0 for index in range(20)}
+ rows = [row] * 100
+ query_results = [
+ {"colnames": list(row), "data": rows, "rowcount": len(rows)},
+ {"colnames": list(row), "data": rows, "rowcount": len(rows)},
+ ]
+ source = {"queries": query_results}
+ source_bytes = len(json.dumps(source, separators=(",", ":")).encode())
+ monkeypatch.setattr(
+ importlib.import_module("superset.mcp_service.chart.query_result"),
+ "MAX_QUERY_RESULT_VALUE_BYTES",
+ source_bytes,
)
- class MultiQueryChartDataCommand:
+ data, source_failure = query_result_data(source)
+ assert source_failure is None
+ assert data == [rows, rows]
+
+ response = _multi_query_chart_data(query_results)
+ assert len(response.model_dump_json().encode()) > source_bytes
+ response_failure = response_json_failure(response)
+
+ assert response_failure is not None
+ assert "response exceeds the total JSON-encoded byte limit" in (
+ response_failure.error
+ )
+
+
+def test_complete_response_preflight_matches_real_pydantic_serialization() -> None:
+ rows = [{"label": 'café "\n', "value": index} for index in range(100)]
+ query_results = [
+ {"colnames": ["label", "value"], "data": rows, "rowcount": len(rows)},
+ {"colnames": ["label", "value"], "data": rows, "rowcount": len(rows)},
+ ]
+ response = _multi_query_chart_data(query_results)
+
+ assert response_json_failure(response) is None
+ assert len(response.model_dump_json().encode()) <= MAX_QUERY_RESULT_VALUE_BYTES
+
+
+@pytest.mark.asyncio
+async def test_unsaved_mixed_timeseries_returns_nonempty_secondary_query(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """The tool response must not collapse a multi-query command result."""
+ from unittest.mock import AsyncMock
+
+ get_data_command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart_data_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_data"
+ )
+
+ class MultiQueryChartDataCommand:
+ def __init__(self, query_context: object) -> None:
+ self.query_context = query_context
+
+ def validate(self) -> None:
+ pass
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "colnames": ["primary"],
+ "data": [],
+ "rowcount": 0,
+ },
+ {
+ "colnames": ["secondary"],
+ "data": [{"secondary": 2}],
+ "rowcount": 1,
+ },
+ ]
+ }
+
+ monkeypatch.setattr(
+ chart_data_module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(
+ get_data_command_module, "ChartDataCommand", MultiQueryChartDataCommand
+ )
+ request = GetChartDataRequest(form_data_key="mixed-chart")
+ response = await _query_from_form_data(
+ {
+ "datasource_id": 1,
+ "datasource_type": "table",
+ "viz_type": "mixed_timeseries",
+ "row_limit": 10,
+ },
+ request,
+ AsyncMock(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.data == []
+ assert response.query_results is not None
+ assert [result.data for result in response.query_results] == [
+ [],
+ [{"secondary": 2}],
+ ]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ "payload",
+ [
+ {"queries": None},
+ {"queries": []},
+ {"queries": [{}]},
+ {"queries": [{"data": None}]},
+ {"queries": [{"data": []}, {}]},
+ {"queries": [{"data": [{"value": 1}], "colnames": ["value"], "coltypes": []}]},
+ ],
+)
+async def test_unsaved_chart_data_rejects_malformed_query_envelopes(
+ monkeypatch: pytest.MonkeyPatch, payload: dict[str, Any]
+) -> None:
+ from unittest.mock import AsyncMock
+
+ chart_data_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_data"
+ )
+ get_data_command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class MalformedChartDataCommand:
def __init__(self, query_context: object) -> None:
self.query_context = query_context
- def validate(self) -> None:
- pass
+ def validate(self) -> None:
+ pass
+
+ def run(self) -> dict[str, Any]:
+ return payload
+
+ monkeypatch.setattr(
+ chart_data_module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(
+ chart_data_module,
+ "event_logger",
+ SimpleNamespace(log_context=lambda **_kwargs: nullcontext()),
+ )
+ monkeypatch.setattr(
+ get_data_command_module, "ChartDataCommand", MalformedChartDataCommand
+ )
+
+ response = await _query_from_form_data(
+ {"datasource_id": 1, "datasource_type": "table"},
+ GetChartDataRequest(form_data_key="malformed"),
+ AsyncMock(),
+ )
+
+ assert isinstance(response, ChartError)
+ assert response.error_type == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+async def test_unsaved_chart_data_accepts_valid_empty_dataset(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ from unittest.mock import AsyncMock
+
+ chart_data_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_data"
+ )
+ get_data_command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ command = MagicMock()
+ command.run.return_value = {"queries": [{"data": [], "colnames": []}]}
+ monkeypatch.setattr(
+ chart_data_module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(
+ chart_data_module,
+ "event_logger",
+ SimpleNamespace(log_context=lambda **_kwargs: nullcontext()),
+ )
+ monkeypatch.setattr(
+ get_data_command_module, "ChartDataCommand", lambda _context: command
+ )
+
+ response = await _query_from_form_data(
+ {"datasource_id": 1, "datasource_type": "table"},
+ GetChartDataRequest(form_data_key="empty"),
+ AsyncMock(),
+ )
+
+ assert isinstance(response, ChartError)
+ assert response.error_type == "NoData"
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ "row",
+ [
+ _HostileResultRow(value=1),
+ {"value": _HostileResultScalar("unsafe")},
+ object(),
+ ],
+)
+async def test_unsaved_get_data_rejects_hostile_rows_and_scalars(
+ monkeypatch: pytest.MonkeyPatch,
+ row: Any,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {"queries": [{"data": [row], "colnames": ["value"]}]}
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_a, **_k: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="hostile"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartError)
+ assert response.error_type == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+async def test_chart_metadata_distinguishes_booleans_from_equal_integers(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ values = (True, 1, False, 0)
+ return {
+ "queries": [
+ {
+ "data": [{"value": value} for value in values],
+ "colnames": ["value"],
+ "coltypes": [GenericDataType.BOOLEAN],
+ "rowcount": len(values),
+ }
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="boolean-identity"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.columns[0].data_type == "boolean"
+ assert response.columns[0].unique_count == 4
+
+
+def test_numeric_metadata_identity_matches_bounded_value_semantics() -> None:
+ equivalent_groups = [
+ [0, -0.0, 0.0, Decimal("0"), Decimal("-0.000")],
+ [1, 1.0, Decimal("1"), Decimal("1.0"), Decimal("1.00")],
+ [0.5, Decimal("0.5"), Decimal("0.50")],
+ [float("inf"), Decimal("Infinity")],
+ [float("-inf"), Decimal("-Infinity")],
+ ]
+
+ for group in equivalent_groups:
+ identities = [_safe_value_identity(value) for value in group]
+ assert len(set(identities)) == 1
+
+ specials = [
+ Decimal("NaN"),
+ Decimal("-NaN42"),
+ Decimal("sNaN"),
+ Decimal("-sNaN7"),
+ float("nan"),
+ ]
+ identities = [_safe_value_identity(value) for value in specials]
+ assert len(set(identities)) == len(specials)
+ assert identities == [_safe_value_identity(value) for value in specials]
+
+
+def test_aware_datetime_and_time_identity_preserves_instant_semantics() -> None:
+ plus_one = timezone(timedelta(hours=1))
+ same_datetimes = [
+ datetime(2024, 1, 1, 12, tzinfo=timezone.utc),
+ datetime(2024, 1, 1, 13, tzinfo=plus_one),
+ ]
+ distinct_datetime = datetime(2024, 1, 1, 12, tzinfo=plus_one)
+ same_times = [
+ datetime_time(12, tzinfo=timezone.utc),
+ datetime_time(13, tzinfo=plus_one),
+ ]
+ distinct_time = datetime_time(12, tzinfo=plus_one)
+
+ assert len({_safe_value_identity(value) for value in same_datetimes}) == 1
+ assert _safe_value_identity(distinct_datetime) != _safe_value_identity(
+ same_datetimes[0]
+ )
+ assert len({_safe_value_identity(value) for value in same_times}) == 1
+ assert _safe_value_identity(distinct_time) != _safe_value_identity(same_times[0])
+
+
+def test_zoneinfo_time_without_offset_uses_python_naive_identity() -> None:
+ zone = ZoneInfo("America/New_York")
+ naive = datetime_time(12, 34, 56, 789)
+ zoned = datetime_time(12, 34, 56, 789, tzinfo=zone)
+ folded = datetime_time(12, 34, 56, 789, tzinfo=zone, fold=1)
+
+ assert zoned.utcoffset() is None
+ assert zoned == naive == folded
+ assert _safe_value_identity(zoned) == _safe_value_identity(naive)
+ assert _safe_value_identity(folded) == _safe_value_identity(naive)
+
+
+def test_zoneinfo_datetime_fold_matches_python_same_zone_semantics() -> None:
+ zone = ZoneInfo("America/New_York")
+ first = datetime(2024, 11, 3, 1, 30, tzinfo=zone, fold=0)
+ second = datetime(2024, 11, 3, 1, 30, tzinfo=zone, fold=1)
+
+ assert first.utcoffset() != second.utcoffset()
+ assert first == second
+ assert hash(first) == hash(second)
+ assert _safe_value_identity(first) == _safe_value_identity(second)
+
+
+def test_zoneinfo_interzone_ambiguity_matches_python_equality_and_hash() -> None:
+ new_york = ZoneInfo("America/New_York")
+ toronto = ZoneInfo("America/Toronto")
+ ambiguous = datetime(2024, 11, 3, 1, 30, tzinfo=new_york, fold=0)
+ same_utc_instant = datetime(2024, 11, 3, 5, 30, tzinfo=timezone.utc)
+ other_zone = datetime(2024, 11, 3, 1, 30, tzinfo=toronto, fold=0)
+ unambiguous = datetime(2024, 11, 3, 3, 30, tzinfo=new_york)
+ unambiguous_utc = datetime(2024, 11, 3, 8, 30, tzinfo=timezone.utc)
+
+ # PEP 495 makes an offset-dependent ambiguous value unequal across zones,
+ # even when one selected fold denotes the same UTC instant.
+ assert ambiguous != same_utc_instant
+ assert ambiguous != other_zone
+ assert _safe_value_identity(ambiguous) != _safe_value_identity(same_utc_instant)
+ assert _safe_value_identity(ambiguous) != _safe_value_identity(other_zone)
+
+ assert unambiguous == unambiguous_utc
+ assert hash(unambiguous) == hash(unambiguous_utc)
+ assert _safe_value_identity(unambiguous) == _safe_value_identity(unambiguous_utc)
+
+
+def test_time_fold_and_fixed_offset_identity_matches_python() -> None:
+ zone = ZoneInfo("America/New_York")
+ zoned_fold_zero = datetime_time(1, 30, tzinfo=zone, fold=0)
+ zoned_fold_one = datetime_time(1, 30, tzinfo=zone, fold=1)
+ naive = datetime_time(1, 30)
+ utc = datetime_time(12, tzinfo=timezone.utc)
+ plus_one = datetime_time(13, tzinfo=timezone(timedelta(hours=1)))
+
+ assert zoned_fold_zero == zoned_fold_one == naive
+ assert hash(zoned_fold_zero) == hash(zoned_fold_one) == hash(naive)
+ assert (
+ _safe_value_identity(zoned_fold_zero)
+ == _safe_value_identity(zoned_fold_one)
+ == _safe_value_identity(naive)
+ )
+ assert utc == plus_one
+ assert hash(utc) == hash(plus_one)
+ assert _safe_value_identity(utc) == _safe_value_identity(plus_one)
+
+
+def test_opaque_timezone_identity_does_not_execute_custom_offset() -> None:
+ class _HostileTimezone(tzinfo):
+ def utcoffset(self, _value: datetime | None) -> timedelta | None:
+ raise AssertionError("custom timezone hook must not execute")
+
+ def dst(self, _value: datetime | None) -> timedelta | None:
+ raise AssertionError("custom timezone hook must not execute")
+
+ def tzname(self, _value: datetime | None) -> str | None:
+ raise AssertionError("custom timezone hook must not execute")
+
+ hostile = _HostileTimezone()
+ value = datetime(2024, 1, 1, 12, tzinfo=hostile)
+ time_value = datetime_time(12, tzinfo=hostile)
+
+ assert _safe_value_identity(value) == _safe_value_identity(value)
+ assert _safe_value_identity(time_value) == _safe_value_identity(time_value)
+
+
+def test_oversized_numeric_identity_declines_unbounded_conversion() -> None:
+ huge_int = 1 << 100_000
+ scale_variant_a = Decimal("1e1000000")
+ scale_variant_b = Decimal("10e999999")
+ huge_decimal = Decimal("1" * 100_000)
+
+ int_identity = _safe_value_identity(huge_int)
+ decimal_identity = _safe_value_identity(huge_decimal)
+
+ assert int_identity[:2] == ("oversized_numeric", "integer")
+ assert decimal_identity[:2] == ("oversized_numeric", "decimal")
+ assert _safe_value_identity(scale_variant_a) == _safe_value_identity(
+ scale_variant_b
+ )
+ assert len(repr(int_identity)) < 200
+ assert len(repr(decimal_identity)) < 200
+
+
+def test_mixed_numeric_unique_count_uses_value_semantics() -> None:
+ values = [
+ 0,
+ -0.0,
+ Decimal("-0.00"),
+ 1,
+ 1.0,
+ Decimal("1.000"),
+ Decimal("sNaN"),
+ Decimal("sNaN"),
+ Decimal("NaN"),
+ float("inf"),
+ Decimal("Infinity"),
+ ]
+
+ assert len({_safe_value_identity(value) for value in values}) == 5
+
+
+def test_chart_column_metadata_bounds_wide_sparse_statistics_work() -> None:
+ from superset.mcp_service.chart.tool.get_chart_data import _build_data_columns
+ from superset.mcp_service.utils.response_utils import (
+ data_column_stats_row_limit,
+ )
+
+ rows: list[dict[str, Any]] = [{} for _ in range(50_000)]
+ raw_columns = [f"column_{index}" for index in range(4096)]
+
+ columns = _build_data_columns(rows, raw_columns, [])
+
+ sampled_rows = data_column_stats_row_limit(len(rows), len(raw_columns))
+ assert len(columns) == len(raw_columns)
+ assert columns[0].statistics == {"sampled_rows": sampled_rows}
+ assert columns[-1].statistics == {"sampled_rows": sampled_rows}
+
+
+@pytest.mark.asyncio
+async def test_unsaved_generic_get_data_returns_finite_decimal_metadata(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ values = [
+ Decimal("1.25"),
+ Decimal("1.250"),
+ 1.25,
+ 0,
+ -0.0,
+ Decimal("-0.00"),
+ ]
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": value} for value in values],
+ "colnames": ["value"],
+ "rowcount": len(values),
+ }
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="decimal"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.columns[0].data_type == "numeric"
+ assert response.columns[0].unique_count == 2
+ assert response.columns[0].sample_values == values[:3]
+
+
+@pytest.mark.asyncio
+async def test_unsaved_get_data_canonicalizes_decimal_nonfinite_at_producer(
+ mcp_server: Any,
+ mock_auth: Any,
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """The public MCP ChartData wire never receives Decimal non-finite tokens."""
+ from fastmcp import Client
+
+ from superset.dataframe import df_to_records
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ form_data = {
+ "datasource_id": 7,
+ "datasource_type": "table",
+ "datasource": "7__table",
+ "viz_type": "table",
+ "all_columns": ["value"],
+ }
+ finite = Decimal("0.10000000000000000001")
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "value": pd.Series(
+ [
+ Decimal("NaN"),
+ Decimal("sNaN"),
+ Decimal("Infinity"),
+ Decimal("-Infinity"),
+ finite,
+ ],
+ dtype=object,
+ )
+ }
+ ),
+ convert_big_integers=False,
+ )
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["value"],
+ "coltypes": [GenericDataType.NUMERIC],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+ monkeypatch.setattr(
+ module, "get_cached_form_data", lambda _key: json.dumps(form_data)
+ )
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(module, "set_query_context_form_data", lambda *_args: None)
+ monkeypatch.setattr(
+ module,
+ "event_logger",
+ SimpleNamespace(log_context=lambda **_kwargs: nullcontext()),
+ )
+ monkeypatch.setattr(module.guest_scope, "is_guest_read", lambda: False)
+
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"form_data_key": "decimal-nonfinite"}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert [row["value"] for row in payload["data"]] == [
+ None,
+ None,
+ None,
+ None,
+ "0.10000000000000000001",
+ ]
+
+
+@pytest.mark.asyncio
+async def test_unsaved_get_data_reports_sampled_all_null_completeness(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ row_count = 10_000
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": float("nan")} for _ in range(row_count)],
+ "colnames": ["value"],
+ "coltypes": [GenericDataType.NUMERIC],
+ "rowcount": row_count,
+ }
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="nulls"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.columns[0].null_count == 5000
+ assert response.columns[0].statistics == {"sampled_rows": 5000}
+ assert response.data_quality == {
+ "completeness": 0.0,
+ "completeness_is_approximate": True,
+ "sampled_rows": 5000,
+ }
+ assert all(row["value"] is None for row in response.data)
+
+
+@pytest.mark.asyncio
+async def test_unsaved_get_data_round_trips_temporal_and_boolean_coltypes(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [
+ {
+ "event_time": datetime(
+ 2024,
+ 1,
+ 1,
+ 2,
+ 3,
+ 4,
+ tzinfo=dateutil_tz.tzoffset(
+ "east", 5 * 3600 + 30 * 60
+ ),
+ ),
+ "clock_time": datetime_time(
+ 2, 3, 4, tzinfo=pytz.FixedOffset(-450)
+ ),
+ "enabled": True,
+ "missing": np.float64("nan"),
+ }
+ ],
+ "colnames": [
+ "event_time",
+ "clock_time",
+ "enabled",
+ "missing",
+ ],
+ "coltypes": [
+ GenericDataType.TEMPORAL,
+ GenericDataType.TEMPORAL,
+ GenericDataType.BOOLEAN,
+ GenericDataType.NUMERIC,
+ ],
+ "rowcount": 1,
+ }
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="typed"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert [column.data_type for column in response.columns] == [
+ "temporal",
+ "temporal",
+ "boolean",
+ "numeric",
+ ]
+ assert response.data == [
+ {
+ "event_time": "2024-01-01T02:03:04+05:30",
+ "clock_time": "02:03:04-07:30",
+ "enabled": True,
+ "missing": None,
+ }
+ ]
+
+
+@pytest.mark.asyncio
+async def test_unsaved_multi_query_consumes_canonical_cached_dttm(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ cached_dttm = (datetime.now(timezone.utc) - timedelta(minutes=90)).isoformat()
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": 1}],
+ "colnames": ["value"],
+ "rowcount": 1,
+ "is_cached": True,
+ "cached_dttm": cached_dttm,
+ },
+ {
+ "data": [{"other": 2}],
+ "colnames": ["other"],
+ "rowcount": 1,
+ "is_cached": True,
+ "cached_dttm": cached_dttm,
+ },
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_args, **_kwargs: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {"datasource": "1__table", "viz_type": "table"},
+ GetChartDataRequest(form_data_key="cached"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartData)
+ assert response.cache_status is not None
+ assert response.cache_status.cache_hit is True
+ assert response.cache_status.cache_age_seconds is not None
+ assert response.cache_status.cache_age_seconds >= 5_399
+ assert response.query_results is not None
+ assert len(response.query_results) == 2
+
+
+@pytest.mark.asyncio
+async def test_saved_get_data_accepts_real_postprocessing_null_and_large_full_sql(
+ mcp_server: Any, mock_auth: Any, app_context: None
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ from tests.unit_tests.mcp_service.chart.query_result_test_utils import (
+ real_compare_command_result,
+ )
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=13,
+ slice_name="Comparison",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+ sql = 'SELECT "chart café"\\n' + "x" * (70 * 1024)
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return real_compare_command_result(sql)
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 13}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["data"][0]["finite"] == 2.5
+ assert any(value is None for value in payload["data"][0].values())
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("engine", ["openpyxl", "xlsxwriter"])
+async def test_real_get_chart_data_excel_stringifies_uuid_with_both_engines(
+ engine: str, mcp_server: Any, mock_auth: Any
+) -> None:
+ import base64
+ import io
+ from contextlib import ExitStack
+ from unittest.mock import patch
+
+ from fastmcp import Client
+ from openpyxl import load_workbook
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ identifier = UUID("12345678-1234-5678-1234-567812345678")
+ chart = SimpleNamespace(
+ id=15,
+ slice_name="UUID export",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"id": identifier}],
+ "colnames": ["id"],
+ "rowcount": 1,
+ }
+ ]
+ }
+
+ with ExitStack() as stack:
+ stack.enter_context(
+ patch.object(module, "find_chart_by_identifier", return_value=chart)
+ )
+ stack.enter_context(
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ )
+ )
+ stack.enter_context(
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ )
+ )
+ stack.enter_context(patch.object(command_module, "ChartDataCommand", _Command))
+ if engine == "xlsxwriter":
+ stack.enter_context(
+ patch.object(
+ module, "_create_excel_with_openpyxl", side_effect=ImportError
+ )
+ )
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {"request": {"identifier": 15, "format": "excel"}},
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["format"] == "excel"
+ workbook = load_workbook(io.BytesIO(base64.b64decode(payload["excel_data"])))
+ assert workbook.active["A2"].value == str(identifier)
+
+
+@pytest.mark.asyncio
+async def test_saved_get_data_uses_production_cache_timestamp_for_insights(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=14,
+ slice_name="Cached producer envelope",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+ cached_dttm = (datetime.now(timezone.utc) - timedelta(hours=2)).isoformat()
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": 1}],
+ "colnames": ["value"],
+ "rowcount": 1,
+ "is_cached": True,
+ "cached_dttm": cached_dttm,
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 14}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["cache_status"]["cache_hit"] is True
+ assert payload["cache_status"]["cache_age_seconds"] >= 7_199
+ assert any("2h old" in insight for insight in payload["insights"])
+ assert payload["performance"]["cache_status"] == "cache_hit"
+
+
+@pytest.mark.asyncio
+async def test_unsaved_bullet_get_data_uses_strict_render_model(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {"data": [{"Revenue": "not numeric"}], "colnames": ["Revenue"]}
+ ]
+ }
+
+ monkeypatch.setattr(
+ module,
+ "build_query_context_from_form_data",
+ lambda *_a, **_k: _query_context_stub(),
+ )
+ monkeypatch.setattr(command_module, "ChartDataCommand", _Command)
+
+ response = await _query_from_form_data(
+ {
+ "datasource": "1__table",
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ },
+ GetChartDataRequest(form_data_key="bullet"),
+ _AsyncContext(),
+ )
+
+ assert isinstance(response, ChartError)
+ assert response.error_type == "MalformedBulletOutput"
+
+
+def _make_chart_data(**overrides: Any) -> ChartData:
+ """Build a minimal valid ChartData for testing."""
+ from superset.mcp_service.common.cache_schemas import CacheStatus
+
+ defaults: dict[str, Any] = {
+ "chart_id": 1,
+ "chart_name": "Test Chart",
+ "chart_type": "table",
+ "columns": [],
+ "data": [{"metric": 3.5}],
+ "row_count": 1,
+ "total_rows": 1,
+ "data_freshness": None,
+ "summary": "summary",
+ "insights": [],
+ "data_quality": {},
+ "recommended_visualizations": [],
+ "performance": PerformanceMetadata(
+ query_duration_ms=42,
+ cache_status="fresh_query",
+ ),
+ "cache_status": CacheStatus(cache_hit=False),
+ }
+ defaults.update(overrides)
+ return ChartData(**defaults)
+
+
+class TestChartDataTotalRowsCoercion:
+ """Regression tests: float total_rows causes PydanticSerializationError."""
+
+ def test_float_total_rows_is_coerced_to_int(self) -> None:
+ chart_data = _make_chart_data(total_rows=5.0)
+ assert chart_data.total_rows == 5
+ assert isinstance(chart_data.total_rows, int)
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ "row",
+ [
+ _HostileResultRow(value=1),
+ {"value": _HostileResultScalar("unsafe")},
+ object(),
+ ],
+)
+async def test_saved_get_data_rejects_hostile_rows_and_scalars(
+ mcp_server: Any,
+ mock_auth: Any,
+ row: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=9,
+ slice_name="Hostile rows",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {"queries": [{"data": [row], "colnames": ["value"]}]}
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 9}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["error_type"] == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+async def test_saved_generic_get_data_rejects_misaligned_coltypes(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=12,
+ slice_name="Malformed metadata",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": 1}],
+ "colnames": ["value"],
+ "coltypes": [],
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 12}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["error_type"] == "MalformedQueryResult"
+
+
+@pytest.mark.asyncio
+async def test_saved_generic_get_data_returns_finite_decimal_metadata(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=11,
+ slice_name="Decimal metadata",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+ values = [
+ Decimal("1.25"),
+ Decimal("1.250"),
+ 1.25,
+ 0,
+ -0.0,
+ Decimal("-0.00"),
+ ]
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [{"value": value} for value in values],
+ "colnames": ["value"],
+ "rowcount": len(values),
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 11}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["columns"][0]["data_type"] == "numeric"
+ assert payload["columns"][0]["unique_count"] == 2
+ assert payload["columns"][0]["sample_values"] == ["1.25", "1.250", 1.25]
+
+
+@pytest.mark.asyncio
+async def test_saved_get_data_round_trips_temporal_and_boolean_coltypes(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=13,
+ slice_name="Typed metadata",
+ viz_type="table",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "table"}',
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": [
+ {
+ "event_time": pd.Timestamp(
+ datetime(
+ 2024,
+ 1,
+ 1,
+ 2,
+ 3,
+ 4,
+ tzinfo=pytz.FixedOffset(-450),
+ )
+ ),
+ "enabled": False,
+ }
+ ],
+ "colnames": ["event_time", "enabled"],
+ "coltypes": [
+ GenericDataType.TEMPORAL,
+ GenericDataType.BOOLEAN,
+ ],
+ "rowcount": 1,
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 13}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert [column["data_type"] for column in payload["columns"]] == [
+ "temporal",
+ "boolean",
+ ]
+ assert payload["data"] == [
+ {"event_time": "2024-01-01T02:03:04-07:30", "enabled": False}
+ ]
+
+
+@pytest.mark.asyncio
+async def test_saved_bullet_get_data_uses_strict_render_model(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=10,
+ slice_name="Strict Bullet",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params='{"viz_type": "bullet", "metric": "Revenue"}',
+ )
+
+ class _Command:
+ def __init__(self, query_context: Any) -> None: ...
+ def validate(self) -> None: ...
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {"data": [{"Revenue": "not numeric"}], "colnames": ["Revenue"]}
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 10}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["error_type"] == "MalformedBulletOutput"
+
+
+@pytest.mark.asyncio
+async def test_saved_bullet_get_data_projects_dataframe_timestamps_to_epoch(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ from superset.dataframe import df_to_records
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ dateutil_dublin = dateutil_tz.gettz("Europe/Dublin")
+ dateutil_new_york = dateutil_tz.gettz("America/New_York")
+ assert dateutil_dublin is not None
+ assert dateutil_new_york is not None
+ dublin_fold = datetime(
+ 2024,
+ 10,
+ 27,
+ 1,
+ 30,
+ 0,
+ 123456,
+ tzinfo=dateutil_dublin,
+ fold=1,
+ )
+ new_york_gap = datetime(2024, 3, 10, 2, 30, 0, 123456, tzinfo=dateutil_new_york)
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=20,
+ slice_name="Timestamp Bullet",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(form_data),
+ )
+ source_values = [
+ pd.Timestamp("2024-01-02 03:04:05.123456789"),
+ pd.Timestamp("2024-01-02 08:34:05.123456789+05:30"),
+ pd.Timestamp(
+ datetime(
+ 2024,
+ 11,
+ 3,
+ 1,
+ 30,
+ tzinfo=ZoneInfo("America/New_York"),
+ fold=0,
+ )
+ ),
+ pd.Timestamp(
+ datetime(
+ 2024,
+ 11,
+ 3,
+ 1,
+ 30,
+ tzinfo=ZoneInfo("America/New_York"),
+ fold=1,
+ )
+ ),
+ pd.Timestamp("1969-12-31 23:59:59.999999999"),
+ pd.NaT,
+ dublin_fold,
+ new_york_gap,
+ datetime(2040, 7, 1, 12, 0, 0, 123456, tzinfo=dateutil_new_york),
+ pd.Timestamp(dublin_fold),
+ pd.Timestamp(new_york_gap),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 20}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert [row["Category"] for row in payload["data"]] == [
+ 1704164645123.456,
+ 1704164645123.456,
+ 1730611800000.0,
+ 1730615400000.0,
+ -0.0010000000000287557,
+ None,
+ 1729989000123.456,
+ 1710052200123.456,
+ 2224774800123.456,
+ 1729989000123.456,
+ 1710052200123.456,
+ ]
+
+
+@pytest.mark.asyncio
+async def test_transitionless_dateutil_dataframe_reaches_fastmcp_data(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from dateutil.zoneinfo import get_zonefile_instance
+ from fastmcp import Client
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+ from superset.utils.json import json_int_dttm_ser
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ names = [
+ "UTC",
+ "GMT",
+ "Universal",
+ "Zulu",
+ "EST",
+ "HST",
+ "MST",
+ "Etc/GMT+1",
+ "Etc/GMT-2",
+ ]
+ values = []
+ for getter in (dateutil_tz.gettz, get_zonefile_instance().get):
+ for name in names:
+ timezone_value = getter(name)
+ assert timezone_value is not None
+ values.append(
+ datetime(2040, 7, 1, 12, 34, 56, 123456, tzinfo=timezone_value)
+ )
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Category": pd.Series(values, dtype=object),
+ "Revenue": range(1, len(values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+
+ class _ProducerContext:
+ result_type = ChartDataResultType.FULL
+
+ def get_payload(self, **_kwargs: Any) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Category", "Revenue"],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ producer_result = ProducerChartDataCommand(
+ _ProducerContext() # type: ignore[arg-type]
+ ).run()
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Category"],
+ }
+ chart = SimpleNamespace(
+ id=22,
+ slice_name="Transitionless timestamps",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(form_data),
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return producer_result
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 22}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["row_count"] == len(values)
+ assert payload["total_rows"] == len(values)
+ assert [row["Category"] for row in payload["data"]] == [
+ json_int_dttm_ser(value) for value in values
+ ]
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ ("format_", "excel_engine"),
+ [
+ ("json", None),
+ ("csv", None),
+ ("excel", "openpyxl"),
+ ("excel", "xlsxwriter"),
+ ],
+)
+async def test_saved_cached_bullet_duration_producer_reaches_fastmcp_exports(
+ mcp_server: Any,
+ mock_auth: Any,
+ format_: str,
+ excel_engine: str | None,
+) -> None:
+ import base64
+ import csv
+ import io
+ from contextlib import ExitStack
+ from unittest.mock import patch
+
+ from fastmcp import Client
+ from openpyxl import load_workbook
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ source_values = [
+ timedelta(0),
+ timedelta(microseconds=-1),
+ timedelta(days=1, seconds=2, microseconds=3),
+ pd.Timedelta(-1, unit="ns"),
+ pd.Timedelta("1 days 00:00:02.000003004"),
+ np.timedelta64(123456789, "ns"),
+ np.timedelta64("NaT"),
+ ]
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Duration": pd.Series(source_values, dtype=object),
+ "Revenue": range(1, len(source_values) + 1),
+ }
+ ),
+ convert_big_integers=False,
+ )
+ assert type(rows[5]["Duration"]) is pd.Timedelta
+ assert rows[6]["Duration"] is None
+ expected = [
+ None
+ if row["Duration"] is None
+ else json.loads(json.dumps(row["Duration"], default=json.json_int_dttm_ser))
+ for row in rows
+ ]
+
+ class _ProducerContext:
+ result_type = ChartDataResultType.FULL
+
+ def get_payload(self, **_kwargs: Any) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Duration", "Revenue"],
+ "rowcount": len(rows),
+ "is_cached": True,
+ "cache_key": "duration-cache",
+ "cached_dttm": "2026-09-03T00:00:00+00:00",
+ }
+ ]
+ }
+
+ producer_result = ProducerChartDataCommand(
+ _ProducerContext() # type: ignore[arg-type]
+ ).run()
+ form_data = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Duration"],
+ }
+ chart = SimpleNamespace(
+ id=23,
+ slice_name="Duration Bullet",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(form_data),
+ )
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return producer_result
+
+ with ExitStack() as stack:
+ stack.enter_context(
+ patch.object(module, "find_chart_by_identifier", return_value=chart)
+ )
+ stack.enter_context(
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ )
+ )
+ stack.enter_context(
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ )
+ )
+ stack.enter_context(patch.object(command_module, "ChartDataCommand", _Command))
+ if excel_engine == "xlsxwriter":
+ stack.enter_context(
+ patch.object(
+ module, "_create_excel_with_openpyxl", side_effect=ImportError
+ )
+ )
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {"request": {"identifier": 23, "format": format_}},
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["row_count"] == len(rows)
+ assert payload["total_rows"] == len(rows)
+ assert payload["cache_status"]["cache_hit"] is True
+ if format_ == "json":
+ assert [row["Duration"] for row in payload["data"]] == expected
+ elif format_ == "csv":
+ decoded = list(csv.DictReader(io.StringIO(payload["csv_data"])))
+ assert [row["Duration"] or None for row in decoded] == expected
+ else:
+ workbook = load_workbook(io.BytesIO(base64.b64decode(payload["excel_data"])))
+ assert [cell.value for cell in workbook.active[1]] == ["Duration", "Revenue"]
+ assert [
+ workbook.active.cell(row=index + 2, column=1).value
+ for index in range(len(expected))
+ ] == expected
+
+
+@pytest.mark.asyncio
+async def test_form_data_key_bullet_duration_producer_uses_chart_data_wire(
+ mcp_server: Any,
+ mock_auth: Any,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ from superset.commands.chart.data.get_data_command import (
+ ChartDataCommand as ProducerChartDataCommand,
+ )
+ from superset.common.chart_data import ChartDataResultType
+ from superset.dataframe import df_to_records
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ rows = df_to_records(
+ pd.DataFrame(
+ {
+ "Duration": pd.Series(
+ [timedelta(days=1, microseconds=3), pd.Timedelta(-1, "ns")],
+ dtype=object,
+ ),
+ "Revenue": [1, 2],
+ }
+ ),
+ convert_big_integers=False,
+ )
+ expected = [
+ json.loads(json.dumps(row["Duration"], default=json.json_int_dttm_ser))
+ for row in rows
+ ]
+
+ class _ProducerContext:
+ result_type = ChartDataResultType.FULL
+
+ def get_payload(self, **_kwargs: Any) -> dict[str, Any]:
+ return {
+ "queries": [
+ {
+ "data": rows,
+ "colnames": ["Duration", "Revenue"],
+ "rowcount": len(rows),
+ }
+ ]
+ }
+
+ producer_result = ProducerChartDataCommand(
+ _ProducerContext() # type: ignore[arg-type]
+ ).run()
+ form_data = {
+ "datasource": "1__table",
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ "groupby": ["Duration"],
+ }
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return producer_result
+
+ with (
+ patch.object(
+ module, "get_cached_form_data", return_value=json.dumps(form_data)
+ ),
+ patch.object(
+ module,
+ "build_query_context_from_form_data",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {"request": {"form_data_key": "duration-bullet"}},
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["chart_id"] == 0
+ assert [row["Duration"] for row in payload["data"]] == expected
- def run(self) -> dict[str, Any]:
- return {
- "queries": [
- {
- "colnames": ["primary"],
- "data": [],
- "rowcount": 0,
- },
- {
- "colnames": ["secondary"],
- "data": [{"secondary": 2}],
- "rowcount": 1,
- },
- ]
+
+GANTT_FORM_DATA: dict[str, Any] = {
+ "viz_type": "gantt_chart",
+ "start_time": "start_time",
+ "end_time": "end_time",
+ "y_axis": "task",
+}
+
+
+@pytest.mark.asyncio
+async def test_saved_empty_gantt_get_data_reports_no_data(
+ mcp_server: Any, mock_auth: Any
+) -> None:
+ """Only Bullet treats a zero-row result as renderable; Gantt reports NoData."""
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ chart = SimpleNamespace(
+ id=22,
+ slice_name="Empty Gantt",
+ viz_type="gantt_chart",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(GANTT_FORM_DATA),
+ )
+ command = MagicMock()
+ command.run.return_value = {
+ "queries": [
+ {
+ "data": [],
+ "colnames": ["start_time", "end_time", "task"],
+ "rowcount": 0,
}
+ ]
+ }
+
+ with (
+ patch.object(module, "find_chart_by_identifier", return_value=chart),
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ ),
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", lambda _context: command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data", {"request": {"identifier": 22}}
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert payload["error_type"] == "NoData"
+
+
+@pytest.mark.asyncio
+async def test_unsaved_empty_gantt_get_data_reports_no_data(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ from unittest.mock import AsyncMock
+ chart_data_module = importlib.import_module(
+ "superset.mcp_service.chart.tool.get_chart_data"
+ )
+ get_data_command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ command = MagicMock()
+ command.run.return_value = {
+ "queries": [{"data": [], "colnames": ["start_time", "end_time", "task"]}]
+ }
monkeypatch.setattr(
chart_data_module,
"build_query_context_from_form_data",
- lambda *_args, **_kwargs: object(),
+ lambda *_args, **_kwargs: _query_context_stub(),
)
monkeypatch.setattr(
- get_data_command_module, "ChartDataCommand", MultiQueryChartDataCommand
+ chart_data_module,
+ "event_logger",
+ SimpleNamespace(log_context=lambda **_kwargs: nullcontext()),
)
- request = GetChartDataRequest(form_data_key="mixed-chart")
+ monkeypatch.setattr(
+ get_data_command_module, "ChartDataCommand", lambda _context: command
+ )
+
response = await _query_from_form_data(
- {
- "datasource_id": 1,
- "datasource_type": "table",
- "viz_type": "mixed_timeseries",
- "row_limit": 10,
- },
- request,
+ {**GANTT_FORM_DATA, "datasource_id": 1, "datasource_type": "table"},
+ GetChartDataRequest(form_data_key="empty-gantt"),
AsyncMock(),
)
- assert isinstance(response, ChartData)
- assert response.data == []
- assert response.query_results is not None
- assert [result.data for result in response.query_results] == [
- [],
- [{"secondary": 2}],
- ]
+ assert isinstance(response, ChartError)
+ assert response.error_type == "NoData"
-def _make_chart_data(**overrides: Any) -> ChartData:
- """Build a minimal valid ChartData for testing."""
- from superset.mcp_service.common.cache_schemas import CacheStatus
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ ("grouped", "format_", "identifier_alias", "excel_engine"),
+ [
+ (False, "json", "id", None),
+ (True, "json", "chart_id", None),
+ (False, "csv", "id", None),
+ (True, "csv", "chart_id", None),
+ (False, "excel", "id", "openpyxl"),
+ (True, "excel", "id", "openpyxl"),
+ (False, "excel", "chart_id", "xlsxwriter"),
+ (True, "excel", "chart_id", "xlsxwriter"),
+ ],
+)
+async def test_saved_empty_bullet_get_data_fastmcp_returns_normalized_rows(
+ mcp_server: Any,
+ mock_auth: Any,
+ grouped: bool,
+ format_: str,
+ identifier_alias: str,
+ excel_engine: str | None,
+) -> None:
+ import base64
+ import io
+ from contextlib import ExitStack
+ from unittest.mock import patch
- defaults: dict[str, Any] = {
- "chart_id": 1,
- "chart_name": "Test Chart",
- "chart_type": "table",
- "columns": [],
- "data": [{"metric": 3.5}],
- "row_count": 1,
- "total_rows": 1,
- "data_freshness": None,
- "summary": "summary",
- "insights": [],
- "data_quality": {},
- "recommended_visualizations": [],
- "performance": PerformanceMetadata(
- query_duration_ms=42,
- cache_status="fresh_query",
- ),
- "cache_status": CacheStatus(cache_hit=False),
+ from fastmcp import Client
+ from openpyxl import load_workbook
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ form_data: dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": "Revenue",
}
- defaults.update(overrides)
- return ChartData(**defaults)
+ if grouped:
+ form_data["groupby"] = ["Region"]
+ raw_columns = ["Revenue", *(["Region"] if grouped else [])]
+ chart = SimpleNamespace(
+ id=21,
+ slice_name="Empty Bullet",
+ viz_type="bullet",
+ datasource_id=1,
+ datasource_type="table",
+ query_context='{"queries": []}',
+ params=json.dumps(form_data),
+ )
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
-class TestChartDataTotalRowsCoercion:
- """Regression tests: float total_rows causes PydanticSerializationError."""
+ def validate(self) -> None: ...
- def test_float_total_rows_is_coerced_to_int(self) -> None:
- chart_data = _make_chart_data(total_rows=5.0)
- assert chart_data.total_rows == 5
- assert isinstance(chart_data.total_rows, int)
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": 2
+ * [
+ {
+ "data": [],
+ "colnames": raw_columns,
+ "rowcount": 0,
+ }
+ ]
+ }
+
+ with ExitStack() as stack:
+ stack.enter_context(
+ patch.object(module, "find_chart_by_identifier", return_value=chart)
+ )
+ stack.enter_context(
+ patch.object(
+ module,
+ "validate_chart_dataset",
+ return_value=SimpleNamespace(is_valid=True, warnings=[], error=None),
+ )
+ )
+ stack.enter_context(
+ patch(
+ "superset.charts.schemas.ChartDataQueryContextSchema.load",
+ return_value=_query_context_stub(),
+ )
+ )
+ stack.enter_context(patch.object(command_module, "ChartDataCommand", _Command))
+ if excel_engine == "xlsxwriter":
+ stack.enter_context(
+ patch.object(
+ module, "_create_excel_with_openpyxl", side_effect=ImportError
+ )
+ )
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {
+ "request": {
+ identifier_alias: 21,
+ "format": format_,
+ }
+ },
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert "error_type" not in payload
+ if format_ == "json":
+ assert payload["data"] == []
+ assert payload["row_count"] == 0
+ assert payload["total_rows"] == 0
+ assert len(payload["query_results"]) == 2
+ for query_result in payload["query_results"]:
+ assert query_result["data"] == []
+ assert query_result["row_count"] == 0
+ assert query_result["total_rows"] == 0
+ elif format_ == "csv":
+ assert payload["format"] == "csv"
+ assert payload["row_count"] == 0
+ assert payload["total_rows"] == 0
+ assert payload["csv_data"] == (
+ "Revenue,Region\r\n" if grouped else "Revenue\r\n"
+ )
+ else:
+ assert payload["format"] == "excel"
+ assert payload["row_count"] == 0
+ assert payload["total_rows"] == 0
+ workbook = load_workbook(io.BytesIO(base64.b64decode(payload["excel_data"])))
+ assert [cell.value for cell in workbook.active[1]] == raw_columns
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize(
+ "grouped",
+ [
+ False,
+ True,
+ ],
+)
+async def test_form_data_key_empty_bullet_fastmcp_returns_normalized_rows(
+ mcp_server: Any,
+ mock_auth: Any,
+ grouped: bool,
+) -> None:
+ from unittest.mock import patch
+
+ from fastmcp import Client
+
+ module = importlib.import_module("superset.mcp_service.chart.tool.get_chart_data")
+ command_module = importlib.import_module(
+ "superset.commands.chart.data.get_data_command"
+ )
+ form_data: dict[str, Any] = {
+ "datasource": "1__table",
+ "viz_type": "bullet",
+ "metric": "Revenue",
+ }
+ if grouped:
+ form_data["groupby"] = ["Region"]
+ raw_columns = ["Revenue", *(["Region"] if grouped else [])]
+
+ class _Command:
+ def __init__(self, _query_context: Any) -> None: ...
+
+ def validate(self) -> None: ...
+
+ def run(self) -> dict[str, Any]:
+ return {
+ "queries": 2
+ * [
+ {
+ "data": [],
+ "colnames": raw_columns,
+ "rowcount": 0,
+ }
+ ]
+ }
+
+ with (
+ patch.object(
+ module, "get_cached_form_data", return_value=json.dumps(form_data)
+ ),
+ patch.object(
+ module,
+ "build_query_context_from_form_data",
+ return_value=_query_context_stub(),
+ ),
+ patch.object(command_module, "ChartDataCommand", _Command),
+ ):
+ async with Client(mcp_server) as client:
+ result = await client.call_tool(
+ "get_chart_data",
+ {
+ "request": {
+ "form_data_key": "empty-bullet",
+ "format": "json",
+ }
+ },
+ )
+
+ payload = json.loads(result.content[0].text)
+ assert "error_type" not in payload
+ assert payload["chart_id"] == 0
+ assert payload["chart_type"] == "bullet"
+ assert payload["row_count"] == 0
+ assert payload["total_rows"] == 0
+ assert payload["data"] == []
+ assert len(payload["query_results"]) == 2
+ for query_result in payload["query_results"]:
+ assert query_result["data"] == []
+ assert query_result["row_count"] == 0
+ assert query_result["total_rows"] == 0
def test_float_total_rows_serializes_without_error(self) -> None:
Review Comment:
fixed in df03f8e50b: the four total_rows tests are back in TestChartDataTotalRowsCoercion.
##########
superset/mcp_service/chart/tool/update_chart.py:
##########
@@ -363,45 +398,88 @@ def _inherited_state_invalid_keys(
return invalid_keys
+def _prune_inherited_query_state(
+ existing_form_data: dict[str, Any],
+ new_form_data: dict[str, Any],
+ config: ChartConfig,
+ dataset_id: int,
+) -> dict[str, Any]:
+ """Discard incompatible inherited roles and their temporal provenance."""
+ invalid_keys = _inherited_state_invalid_keys(
+ existing_form_data, new_form_data, config, dataset_id
+ )
+ if "adhoc_filters" in invalid_keys:
+ invalid_keys.add(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ return {
+ key: value
+ for key, value in existing_form_data.items()
+ if key not in invalid_keys
+ }
+
+
def _build_replacement_form_data(
existing_form_data: dict[str, Any],
parsed_config: ChartConfig,
effective_dataset_id: int | None,
replacement_dataset_id: int | None = None,
) -> dict[str, Any]:
"""Map and merge a replacement config for preview and save paths."""
+ existing_plugin = plugin_for_viz_type(existing_form_data.get("viz_type"))
+ include_disabled = bool(
+ existing_plugin and existing_plugin.chart_type == parsed_config.chart_type
+ )
new_form_data = map_config_to_form_data(
- parsed_config, dataset_id=effective_dataset_id, include_disabled=True
+ parsed_config,
+ dataset_id=effective_dataset_id,
+ include_disabled=include_disabled,
)
new_form_data.pop("_mcp_warnings", None)
dataset_rebind = replacement_dataset_id is not None
- config_plugin = get_registry().get(parsed_config.chart_type, include_disabled=True)
+ config_plugin = get_registry().get(
+ parsed_config.chart_type, include_disabled=include_disabled
+ )
if replacement_dataset_id is not None and not (
config_plugin is not None and config_plugin.strict_dataset_rebind
):
# Drop only the inherited state the replacement dataset cannot
# resolve, then merge as a same-dataset update. Plugins with a strict
# rebind contract handle the rebind in merge_update_form_data.
- invalid_keys = _inherited_state_invalid_keys(
- existing_form_data,
- new_form_data,
- parsed_config,
- replacement_dataset_id,
+ existing_form_data = _prune_inherited_query_state(
+ existing_form_data, new_form_data, parsed_config, replacement_dataset_id
)
- existing_form_data = {
- key: value
- for key, value in existing_form_data.items()
- if key not in invalid_keys
- }
dataset_rebind = False
merge_table_column_config(existing_form_data, new_form_data)
merge_interactive_pivot_ui_config(existing_form_data, new_form_data)
- merged = _merge_replacement_config(
- existing_form_data,
- new_form_data,
- parsed_config,
- dataset_rebind=dataset_rebind,
- )
+ merge_plugin = plugin_for_viz_type(new_form_data.get("viz_type"))
+ if merge_plugin is not None and merge_plugin.owns_update_merge:
+ # The plugin owns its complete merge contract, including temporal
+ # predicates and dataset rebinds. Generic update merges would restore
+ # inherited state that the contract deliberately removed.
+ merged = _merge_replacement_config(
+ existing_form_data,
+ new_form_data,
+ parsed_config,
+ dataset_rebind=dataset_rebind,
+ )
+ else:
+ merged = _merge_replacement_config(
+ existing_form_data,
+ new_form_data,
+ parsed_config,
+ dataset_rebind=dataset_rebind,
+ )
+ merge_update_form_data(existing_form_data, merged, parsed_config)
Review Comment:
fixed in df03f8e50b: filter provenance merge now returns early across a viz-type change, so the previous Table predicates are not restored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4180329872
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,554 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Bounded container dimensions retain their wire values and frontend category text, including 1,2 for this array. Regression coverage is included.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1119,357 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = value.split(",")
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str:
+ token = token.strip(_JAVASCRIPT_WHITESPACE)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = (
+ _javascript_numeric_string(token)
+ if type(token) is str
+ else float(token)
+ )
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ metric_reference = "".join(
+ "\\" + char if char in ".[]\\" else char for char in model.metric_field
+ )
+ category_field = _unique_bullet_category_field(model.rows)
+ row_field = _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_row", (category_field,)
+ )
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field, row_field)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[row_field] = row_index
+ copied[category_field] = (
+ ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ if model.dimensions
+ else ""
+ )
+ if (
+ range_tooltip_field is not None
+ and containing_range_labels[row_index] is not None
+ ):
+ copied[range_tooltip_field] = containing_range_labels[row_index]
+ values.append(copied)
+
+ # Explore uses indexed rows even when their display labels are identical.
+ category_labels = [row[category_field] for row in values]
+ vega_format = {
+ "SMART_NUMBER": "~s",
+ "SMART_NUMBER_SIGNED": "+~s",
+ }.get(model.y_axis_format, model.y_axis_format)
+ y_encoding = {
+ "field": row_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ "sort": None,
+ "axis": {"labelExpr": f"{json.dumps(category_labels)}[datum.value]"},
+ }
+ tooltip = [
+ {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ },
+ {
+ "field": metric_reference,
+ "type": "quantitative",
+ "title": model.metric_field,
+ "format": vega_format,
+ },
+ ]
+ if range_tooltip_field is not None:
+ tooltip.append(
+ {"field": range_tooltip_field, "type": "nominal", "title": "Range"}
+ )
+ axis_min = min(
+ 0.0,
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ axis_max = max(
+ [*model.measures, *model.ranges, *model.markers, *model.marker_lines]
+ )
+ if axis_min == axis_max:
+ axis_max = axis_min + (abs(axis_min) or 1)
+
+ def label_at(labels: list[str], index: int, value: float, prefix: str) -> str:
+ if index < len(labels) and labels[index]:
+ return labels[index]
+ return (
+ ""
+ if prefix == "Range"
+ else _format_bullet_number(model.y_axis_format, value)
+ )
+
+ def legend_color(name: str) -> dict[str, Any]:
+ return {
+ "datum": name,
+ "type": "nominal",
+ "legend": {"title": None} if model.show_legend else None,
+ }
+
+ layers: list[dict[str, Any]] = []
+ range_entries = sorted(
+ [
+ (
+ threshold,
+ label_at(model.range_labels, index, threshold, "Range"),
+ )
+ for index, threshold in enumerate(model.ranges)
+ ],
+ key=lambda entry: entry[0],
+ reverse=True,
+ )
+ for index, (threshold, label) in enumerate(range_entries):
+ layers.append(
+ {
+ "mark": {
+ "type": "rect",
+ "opacity": max(0.08, 0.28 - index * 0.04),
+ },
+ "encoding": {
+ "x": {"datum": axis_min, "type": "quantitative"},
+ "x2": {"datum": threshold},
+ "y": y_encoding,
+ "color": legend_color(
+ (f"{label}: " if label else "")
+ + f"≤ {_format_bullet_number(model.y_axis_format, threshold)}"
+ ),
+ "tooltip": [
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Range, marker, and marker-line tooltips reference calculated fields. All three tooltip arrays pass the Vega-Lite v5 schema. Regression coverage is included.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,554 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ from superset.utils.number_format import D3_FORMAT_RE
+
+ # Specifier length does not bound its requested output precision. Check the
+ # parsed precision before the formatter can allocate or round any value.
+ match = D3_FORMAT_RE.match(format_)
+ if match and match.group(8) is not None and int(match.group(8)) > 20:
+ raise BulletOutputError(
+ "Bullet number format precision must not exceed 20",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Bullet formatting uses sufficient local Decimal precision for bounded formats without changing the caller context. Regression coverage is included.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4175283467
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1091,350 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = float(token)
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. Saved Bullet range/marker tokens use the JavaScript Number string grammar, including unsigned hex/binary/octal, decimal/exponent forms, and ECMAScript trim characters. Underscores, signed radix literals, Python-only inf/whitespace, and non-ASCII digits are ignored. Regressions cover ranges, markers, and marker_lines; finite-value safety bounds remain unchanged. Frontend reference: plugins/plugin-chart-echarts/src/Bullet/utils.ts, tokenizeToNumericArray.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1750,544 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
+ (form_data.get("all_columns") or [])
+ if raw_mode
+ else (form_data.get("groupby") or [])
+ )
+ table_metrics = [] if raw_mode else list(form_data.get("metrics") or [])
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ query["columns"] = [
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. Table selects only the first eligible temporal column, wraps that one BASE_AXIS, and places it first, matching plugins/plugin-chart-table/src/buildQuery.ts. Regressions cover two temporal columns with and without a preceding non-temporal dimension.
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,431 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
+ viz_type = form_data.get("viz_type")
+ if strict_all_form_refs and (
+ viz_type == "mixed_timeseries"
+ or (
+ isinstance(viz_type, str)
+ and (
+ viz_type.startswith("echarts_timeseries") or viz_type == "echarts_area"
+ )
+ )
+ ):
+ if (raw_x_axis := form_data.get("x_axis")) is not None and (
+ error := column_error(raw_x_axis, "form-data x_axis column")
+ ):
+ return error
+ metric_fields = [
+ "metrics",
+ "size",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ ]
+ if viz_type == "mixed_timeseries":
+ metric_fields.extend(
+ [
+ "metrics_b",
+ "size_b",
+ "timeseries_limit_metric_b",
+ "series_limit_metric_b",
+ ]
+ )
+ for field_name in metric_fields:
+ raw_value = form_data.get(field_name)
+ values = raw_value if isinstance(raw_value, list) else [raw_value]
+ for value in values:
+ if value is not None and (
+ error := metric_error(value, f"form-data {field_name} metric")
+ ):
+ return error
+
+ layer_suffixes = ("", "_b") if viz_type == "mixed_timeseries" else ("",)
+ for suffix in layer_suffixes:
+ sort_field = f"x_axis_sort{suffix}"
+ if sort_field not in form_data or form_data.get(sort_field) is None:
+ continue
+ x_axis = form_data.get(f"x_axis{suffix}", form_data.get("x_axis"))
+ allowed_labels: set[str] = set()
+ if x_axis_label := _native_column_label(x_axis):
+ allowed_labels.add(x_axis_label)
+ raw_metrics = form_data.get(f"metrics{suffix}")
+ for metric in raw_metrics if isinstance(raw_metrics, list) else []:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ raw_limit_metric = form_data.get(f"timeseries_limit_metric{suffix}")
+ limit_metrics = (
+ raw_limit_metric
+ if isinstance(raw_limit_metric, list)
+ else [raw_limit_metric]
+ )
+ for metric in limit_metrics:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ sort_value = form_data[sort_field]
+ if not isinstance(sort_value, str) or sort_value not in allowed_labels:
+ return _native_validation_error(sort_field, repr(sort_value)[:200])
+
+ if (
+ strict_all_form_refs
+ and isinstance(viz_type, str)
+ and viz_type.startswith("deck_")
+ ):
+ # Deck layers store most query roles outside common columns/metrics.
+ # Validate every renderer-consumed raw control as well as the generated
+ # QueryObject so a dataset-only rebind cannot hide or discard a stale
+ # tooltip, cross-filter, spatial, path, or metric reference.
+ for spatial_field in ("spatial", "start_spatial", "end_spatial"):
+ spatial = form_data.get(spatial_field)
+ if spatial is None:
+ continue
+ if not isinstance(spatial, dict):
+ return _native_validation_error(
+ f"form-data {spatial_field}", repr(spatial)[:200]
+ )
+ spatial_type = spatial.get("type")
+ if not isinstance(spatial_type, str):
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ role_fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if role_fields is None:
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ for role_field in role_fields:
+ spatial_value = spatial.get(role_field)
+ if spatial_value is None:
+ return _native_validation_error(
+ f"form-data {spatial_field}.{role_field} column", "missing"
+ )
+ if error := column_error(
+ spatial_value, f"form-data {spatial_field}.{role_field} column"
+ ):
+ return error
+
+ for field_name in (
+ "line_column",
+ "geojson",
+ "dimension",
+ "cross_filter_column",
+ ):
+ column_value = form_data.get(field_name)
+ if column_value is not None and (
+ error := column_error(column_value, f"form-data {field_name} column")
+ ):
+ return error
+
+ tooltip_contents = form_data.get("tooltip_contents")
+ if tooltip_contents is not None and not isinstance(tooltip_contents, list):
+ return _native_validation_error(
+ "form-data tooltip_contents", repr(tooltip_contents)[:200]
+ )
+ for index, item in enumerate(tooltip_contents or []):
+ tooltip_value: Any = None
+ if isinstance(item, str):
+ tooltip_value = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ tooltip_value = item.get("column_name")
+ if tooltip_value is not None and (
+ error := column_error(
+ tooltip_value, f"form-data tooltip_contents[{index}] column"
+ )
+ ):
+ return error
+
+ metric_values: list[tuple[str, Any]] = []
+ if viz_type not in {"deck_geojson", "deck_polygon"}:
+ for field_name in ("metrics", "metric", "size"):
+ raw_deck_metrics = form_data.get(field_name)
+ deck_metrics = (
+ raw_deck_metrics
+ if isinstance(raw_deck_metrics, list)
+ else [raw_deck_metrics]
+ )
+ metric_values.extend(
+ (f"form-data {field_name} metric", deck_metric)
+ for deck_metric in deck_metrics
+ if deck_metric is not None
+ )
+ if viz_type == "deck_polygon" and form_data.get("metric") is not None:
+ metric_values.append(("form-data metric metric", form_data.get("metric")))
+ fixed_metric_fields = (
+ ("point_radius_fixed",)
+ if viz_type in {"deck_scatter", "deck_polygon"}
+ else ()
+ ) + (("line_width",) if viz_type == "deck_path" else ())
+ for field_name in fixed_metric_fields:
+ fixed_value = form_data.get(field_name)
+ deck_metric: Any = (
+ fixed_value
+ if (
+ isinstance(fixed_value, str)
+ and fixed_value
+ and viz_type != "deck_polygon"
+ )
+ else None
+ )
+ if isinstance(fixed_value, dict) and fixed_value.get("type") == "metric":
+ deck_metric = fixed_value.get("value")
+ if deck_metric is not None:
+ metric_values.append((f"form-data {field_name} metric", deck_metric))
+ if viz_type == "deck_path" and form_data.get("breakpoint_metric") is not None:
+ metric_values.append(
+ (
+ "form-data breakpoint_metric metric",
+ form_data.get("breakpoint_metric"),
+ )
+ )
+ for role, deck_metric in metric_values:
+ if error := metric_error(deck_metric, role):
+ return error
+
+ for filter_ in form_data.get("adhoc_filters") or []:
+ if not isinstance(filter_, dict) or filter_.get("expressionType") != "SIMPLE":
+ continue
+ if not strict_all_form_refs and _is_inert_adhoc_filter(filter_):
+ continue
+ subject = filter_.get("subject")
+ clause = str(filter_.get("clause") or "WHERE").upper()
+ if clause == "HAVING" and isinstance(subject, str):
+ metric_matches = [
+ name for name in saved_metrics if name.casefold() == subject.casefold()
+ ]
+ if len(metric_matches) == 1:
+ continue
+ if subject is not None and (
+ error := column_error(subject, "form-data filter column")
+ ):
+ return error
+ if filter_.get("operator") == "TEMPORAL_RANGE" and isinstance(subject, str):
+ try:
+ temporal = resolve_dataset_column(subject, dataset_context)
+ except ValueError:
+ temporal = None
+ if temporal is not None and not temporal.get("is_temporal", False):
+ return _native_validation_error("temporal filter column", subject)
+
+ temporal_lookup = form_data.get("temporal_columns_lookup")
+ if isinstance(temporal_lookup, dict):
+ for column, enabled in temporal_lookup.items():
+ if enabled and (error := column_error(column, "temporal lookup column")):
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. Native validation treats temporal_columns_lookup as datasource metadata, not selected roles; selected physical form/query references remain validated. The regression exercises the real Table replacement/rebind merge and validator with an unused old date absent from the replacement dataset.
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1875 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
from decimal import Decimal
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
return None
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. Trusted extended-precision NumPy floats use round-trippable decimal strings rather than binary64 conversion. The regression passes real object-column DataFrame records through query-result validation and verifies both 1.000000000000000001 precision and finite 1e400 range; this shared path feeds chart, dataset, table responses and exports.
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1338,6 +1340,51 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates; omitted label controls preserve the saved state.
+
+Saved native `ranges`, `markers`, and `marker_lines` controls ignore empty,
+non-numeric, and NaN tokens, matching Explore. If no numeric range remains, the
+preview uses the default band up to 110% of the largest measure. Unrelated
+updates preserve these saved controls. Newly authored typed lists must contain
+finite numbers; infinite native values and oversized controls remain errors.
+
+On same-dataset chart updates, `filters: []` clears both user filters and the
+generated dashboard-time binding. `temporal_column: null` clears only that
+generated binding, preserving user filters. Omitting those controls preserves
+the saved binding. Bullet Vega previews keep separate indexed rows even when
+dimension display labels are identical, and support both `SMART_NUMBER` and
+`SMART_NUMBER_SIGNED` number formats.
+
+### Chart query result limits
+
+MCP chart tools accept up to 50,000 rows per query. The row-shaped `indexnames`
Review Comment:
Fixed in b92dc4189fbb416c0bf2d714524a216c787c2782. The MCP guide and UPDATING.md document the non-configurable 64 KiB per-cell and 16 MiB aggregate JSON caps, UTF-8/binary accounting, affected query/preview/compile/export paths, errors, and their independence from the response-size guard. A pre-fix documentation assertion failed; the existing cell and aggregate boundary regressions remain unchanged and pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5979373906
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4155391823
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1090,342 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = float(token)
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ category_field = _unique_bullet_category_field(model.rows)
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field,)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[category_field] = (
Review Comment:
The derived category text is also the nominal y-axis key, so distinct grouped rows such as `Region = NULL` and `Region = "null"` both become `"null"` and their bars overlap in the Vega preview, while Explore keeps separate indexed rows. Could the row identity be kept separate from the display label?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1090,342 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = float(token)
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ category_field = _unique_bullet_category_field(model.rows)
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field,)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[category_field] = (
+ ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ if model.dimensions
+ else ""
+ )
+ if (
+ range_tooltip_field is not None
+ and containing_range_labels[row_index] is not None
+ ):
+ copied[range_tooltip_field] = containing_range_labels[row_index]
+ values.append(copied)
+
+ y_encoding = {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ "sort": None,
+ }
+ tooltip = [
+ {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ },
+ {
+ "field": model.metric_field,
+ "type": "quantitative",
+ "format": (
+ "~s" if model.y_axis_format == "SMART_NUMBER" else model.y_axis_format
+ ),
+ },
+ ]
+ if range_tooltip_field is not None:
+ tooltip.append(
+ {"field": range_tooltip_field, "type": "nominal", "title": "Range"}
+ )
+ axis_min = min(
+ 0.0,
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ axis_max = max(
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ if axis_min == axis_max:
+ axis_max = axis_min + (abs(axis_min) or 1)
+ vega_format = "~s" if model.y_axis_format == "SMART_NUMBER" else model.y_axis_format
Review Comment:
`SMART_NUMBER_SIGNED` passes the preview format validator, but this forwards it unchanged as a Vega/D3 axis and tooltip format, where it is an invalid format specifier. A saved Bullet using that supported Explore format therefore produces a preview that cannot render; could both format projections translate the signed pseudo-format too?
##########
superset/mcp_service/chart/tool/update_chart_preview.py:
##########
@@ -243,62 +294,75 @@ def update_chart_preview( # noqa: C901
if previous_form_data:
merge_table_column_config(previous_form_data, new_form_data)
merge_interactive_pivot_ui_config(previous_form_data, new_form_data)
- new_form_data = merge_chart_form_data(
- previous_form_data,
- new_form_data,
- config,
- dataset_rebind=dataset_rebind,
- )
+ merge_plugin = plugin_for_viz_type(new_form_data.get("viz_type"))
+ if merge_plugin is not None and merge_plugin.owns_update_merge:
+ # The plugin owns its temporal/rebind merge contract.
+ # Generic merges would restore deliberately removed state.
+ new_form_data = merge_chart_form_data(
+ previous_form_data,
+ new_form_data,
+ config,
+ dataset_rebind=dataset_rebind,
+ )
+ else:
+ new_form_data = merge_chart_form_data(
+ previous_form_data,
+ new_form_data,
+ config,
+ dataset_rebind=dataset_rebind,
+ )
+ merge_update_form_data(previous_form_data, new_form_data, config)
+ merge_same_viz_form_data(previous_form_data, new_form_data, config)
Review Comment:
On a cached-preview dataset change, `merge_chart_form_data` first strips the old query state, but this generic preservation pass copies missing keys from the previous dataset back in, including its `datasource` and omitted grouping columns. The returned `form_data` can name the old dataset while the Explore link uses the new one, or validation fails on an old grouping column the caller never supplied. Could dataset rebinds avoid restoring those old query/datasource keys?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1781,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+ """
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
Review Comment:
A saved chart rebind to a dataset without its old time column removes the inherited temporal filter in `_build_replacement_form_data`, but leaves `_mcp_dashboard_time_filter_subject` behind. This lookup then raises `found 0`, so an otherwise valid Table/Bullet config and dataset replacement fails before saving. Could the provenance marker be discarded alongside the invalid inherited filter during a rebind?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the selected roles. Its zero-valued
+ # ungrouped row is a render-only frontend fallback, not source query
+ # data, so an empty query exposes no rows to get-data and exports.
+ return (model.rows if data else []), None
+
+ def ascii_preview(
+ self, data: list[Any], form_data: dict[str, Any], width: int
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def vega_lite_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> VegaLitePreview | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_bullet_vega_lite_preview,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_bullet_vega_lite_preview(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def resolve_update_config(
+ self,
+ config: Any,
+ existing_form_data: dict[str, Any],
+ *,
+ dataset_rebind: bool,
+ ) -> Any:
+ """Resolve sort targets against the saved hierarchy before mapping."""
+ if not isinstance(config, BulletChartConfig) or config.dimensions is not None:
+ return config
+ if not config.order_by:
+ return config
+ dimensions = (
+ existing_form_data.get("groupby") or []
+ if not dataset_rebind and existing_form_data.get("viz_type") == "bullet"
+ else []
+ )
+ return BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": dimensions}
+ )
+
+ def merge_update_form_data(
+ self,
+ existing_form_data: dict[str, Any],
+ new_form_data: dict[str, Any],
+ config: Any,
+ *,
+ dataset_rebind: bool,
+ ) -> dict[str, Any] | None:
+ """Merge filter provenance and preserve omitted native Bullet controls."""
+ if not isinstance(config, BulletChartConfig) or dataset_rebind:
+ return None
Review Comment:
A cached-preview update that switches datasets drops the saved Bullet hierarchy here when `dimensions` is omitted, even if the new dataset has the same grouping columns; the saved-chart update path instead checks those references and preserves them. The same metric/dataset update can therefore turn a grouped Bullet into one aggregate only in the preview flow. Could compatible omitted dimensions follow the same preservation contract on both paths?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4129947280
##########
superset/mcp_service/chart/tool/update_chart.py:
##########
@@ -72,6 +78,13 @@
logger = logging.getLogger(__name__)
+def _bounded_update_response(payload: object) -> GenerateChartResponse:
Review Comment:
Fixed in 70611d12b4f566cd690093318d5159cf28caa57c: forwarded the persisted chart ID to response preflight after a saved update, with update-specific confirmation and retry guidance. The oversized-response regression verifies that the saved chart ID remains available.
##########
superset/mcp_service/chart/tool/generate_chart.py:
##########
@@ -85,6 +98,10 @@ async def generate_chart( # noqa: C901
- Set save_chart=True to permanently save the chart
- LLM clients MUST display returned chart URL to users
- Use numeric dataset ID or UUID (NOT schema.table_name format)
+ - MUST include chart_type in config (one of: 'xy', 'table', 'pie', 'bullet',
Review Comment:
Fixed in 70611d12b4f566cd690093318d5159cf28caa57c: removed the duplicate discriminator list and added bullet to the existing correct list, retaining gauge, treemap_v2, bubble_v2, and gantt.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5884979736
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4244870480
##########
superset/mcp_service/chart/validation/pipeline.py:
##########
@@ -157,6 +156,34 @@ def validate_request_with_warnings(
error_code="GANTT_SEMANTIC_VALIDATION_ERROR",
),
)
+ except ValueError as ex:
Review Comment:
The test name and docstring promise that the pipeline fails closed on an ambiguous Bullet reference, but it only calls `_normalize_column_names` directly. This new `except ValueError` branch (the `AMBIGUOUS_COLUMN_REFERENCE` result) is never exercised: with `Region`/`region` both in the dataset and a dimension of `rEgIoN`, removing this handler would surface `validation_system_error` and the current test would still pass. Could you add a `validate_request_with_warnings` case asserting `is_valid is False` and `error_code == "AMBIGUOUS_COLUMN_REFERENCE"`?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -3538,7 +3570,660 @@ def _as_column_list(value: Any) -> list[Any]:
return [value]
-def analyze_chart_capabilities(viz_type: str | None, config: Any) -> ChartCapabilities:
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ # A missing saved viz_type is a boundary too, matching merge_chart_form_data.
+ if existing_form_data.get("viz_type") != new_form_data.get("viz_type"):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+ if (
+ existing_binding is None
+ and existing_subject is None
+ and range_explicit
+ and not subject_authoritative
+ and "filters" not in explicit_fields
+ ):
+ # A range-only update on an unmarked chart adopts its saved temporal
+ # filter even when Explore bound it to a column other than the mapper's
+ # proposed subject. Zero or several native ranges stay ambiguous, so the
+ # proposed binding is added instead of guessing which one to rewrite.
+ unmarked_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("expressionType") == "SIMPLE"
+ and filter_.get("clause") == "WHERE"
+ and isinstance(filter_.get("subject"), str)
+ ]
+ if len(unmarked_matches) == 1:
+ existing_binding = unmarked_matches[0]
+ existing_subject = existing_binding["subject"]
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ elif range_explicit:
+ # With no saved binding to rewrite, an explicit range uses the
+ # mapper's generated binding rather than being dropped.
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+# The shared preservation registry may carry envelope, presentation, and time
+# keys over from another visualization. Bullet does not model them, so they are
+# opaque persisted state rather than misspelled typed fields.
+_BULLET_UNMODELED_PRESERVED_KEYS = (
+ FORM_DATA_UPDATE_PRESERVE_KEYS["envelope"]
+ | FORM_DATA_UPDATE_PRESERVE_KEYS["presentation"]
+ | FORM_DATA_UPDATE_PRESERVE_KEYS["time"]
+) - _get_known_fields(BulletChartConfig)
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet controls without reinterpreting inherited query roles.
+
+ Saved Explore state may contain SQL dimensions and SQL WHERE/SIMPLE HAVING
+ filters beyond the typed authoring surface. Omitted roles are validated by
+ the native query contract and compilation, not as newly authored physical
+ columns or SIMPLE WHERE filters. Only this validation copy excludes them
+ and native query metadata; compiled and persisted form data stays intact.
+ Explicit replacements, including ``[]``, retain strict typed validation.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ for native_query_key in ("url_params", "extra_form_data", "extra_filters"):
+ validation_data.pop(native_query_key, None)
+ for preserved_key in _BULLET_UNMODELED_PRESERVED_KEYS:
+ validation_data.pop(preserved_key, None)
+ if not validation_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT):
+ # A null marker records an explicit subject clear with no binding to
+ # validate; it is not an authored physical column.
+ validation_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+ if update_config is None or isinstance(update_config, BulletChartConfig):
+ if update_config is None or update_config.dimensions is None:
+ validation_data.pop("groupby", None)
Review Comment:
A partial Bullet update can swap the measure for a dimension. For a saved Bullet grouped by `year`, updating only the metric to `{"name":"revenue","aggregate":"SUM","label":"year"}` (no `dimensions`) drops the saved `groupby` from this validation copy, so the metric-vs-dimension output-label collision check never sees the inherited hierarchy. The query then has two outputs labeled `year`, and the preview/saved chart can show the year values instead of `SUM(revenue)` without any error. Should the collision check run against the final merged hierarchy (e.g. via `_inherited_groupby`) before compiling?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5498318315
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 9cdaada4a526504cb03139557394132c13e15845 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6a97150840ff79000756dc4f |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-43770--superset-docs-preview.netlify.app](https://deploy-preview-43770--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#q1YqKMrPLShRslJS0lFKKkrMS84AsouTdQ0NLQ1NzXVzkwt0k0pzclJLgPIpqQU5-ZWeKUAVZomW5oamBhYmBmlp5pYGBgbmpmYpySZpSrUA) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-43770--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5504089265
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5506076008
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3906690314
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1065,6 +1066,155 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ return ",".join(
+ format(value, "g") if isinstance(value, float) else value for value in values
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Float precision loss</b></div>
<div id="fix">
`format(value, "g")` rounds floats to 6 significant digits (e.g. `123456.789` → `'123457'`). The frontend `tokenizeToNumericArray` in `Bullet/utils.ts` parses these tokens back with `Number(token)`, so bullet range/marker thresholds are silently altered. Use `str(value)` to preserve precision.
</div>
</div>
<small><i>Code Review Run #658aec</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4193994152
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2445,92 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding exact output keys avoids splitting category values that contain
+ escaped commas. The legend retains each complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ escape_separator(label)
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(label + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(label + "__")
+ for label in metric_labels
+ )
+ ]
+ if not fields:
+ return None
+ sample = data[0][x_axis]
+ x_type = (
+ "temporal"
+ if isinstance(sample, str) and any(char in sample for char in "-/: ")
+ else "quantitative"
+ if isinstance(sample, (int, float))
+ else "nominal"
+ )
+ return VegaLitePreview(
+ specification={
+ "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
+ "data": {"values": data},
+ "transform": [
+ {"fold": fields, "as": ["__mcp_xy_series", "__mcp_xy_value"]}
Review Comment:
The pivot output keys in `fields` are passed to `fold` unescaped, and Vega treats `.` and `[` in a field name as a nested lookup. A grouped XY chart with a category like `US.East` produces the literal key `SUM(revenue), US.East`, so that series is dropped from the preview while the call reports success. Should the fold fields be escaped (e.g. `US\.East`) while the data keys stay literal, with a test covering a dotted category?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -286,7 +2244,8 @@ def normalize_chart_query_result(result: Any, form_data: Mapping[str, Any]) -> A
def normalize_treemap_query_result(result: Any, form_data: Mapping[str, Any]) -> Any:
"""Require unique hierarchy outputs and finite numeric Treemap metrics."""
- if failure := query_result_failure(result):
+ _data, failure = query_result_data(result, preserve_nonfinite_floats=True)
Review Comment:
This call rewrites the result rows in place (datetimes become ISO text) before `get_chart_data` runs its format-aware pass, so a Treemap grouped by a datetime column exports `2026-10-05T12:00:00` in CSV instead of the established `2026-10-05 12:00:00`, and Excel gets text cells instead of date cells. Could validation here avoid mutating `result`, or run on a copy / after the export-specific normalization?
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1349,6 +1362,118 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Inherited native SQL dimensions remain query expressions when updating metrics
+or `order_by`; reference their output labels to sort by them. Caller-supplied
+`dimensions` must still be physical columns, not SQL expressions.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling. Bullet metric strings use
+JavaScript numeric spellings: underscore separators and non-ASCII digits are
+rejected rather than interpreted as numbers. Bounded array-valued dimensions
+retain their raw values in data reads and exports; preview category labels use
+JavaScript string conversion (for example, `[1, 2]` displays as `1,2`). Native
+SQL metrics without a label use their SQL expression as the output label.
+
+Presentation-only updates preserve saved predicates, including legacy top-level
+`where`, `having`, and `filters`, and normalize the native `order_by_cols` alias.
+When both `orderby` and `order_by_cols` are saved, their sort entries are
+concatenated in form-data key order, including when `orderby` is empty.
+Explicit `filters: []` and `order_by: []` clear those inherited controls.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+Labels that become empty after sanitization retain their slots so later labels
+remain aligned with their corresponding values.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates. Omitted label controls preserve the saved state only when their
+corresponding value controls (`ranges`, `markers`, or `marker_lines`) are also
+omitted. Replacing a value control without its labels clears the saved labels;
+resend the labels to retain annotations with the replacement values.
+
+Saved native `ranges`, `markers`, and `marker_lines` controls ignore empty,
+non-numeric, and NaN tokens, matching Explore. If no numeric range remains, the
+preview uses the default band up to 110% of the largest measure. Unrelated
+updates preserve these saved controls. Newly authored typed lists must contain
+finite numbers; infinite native values and oversized controls remain errors.
+
+On same-dataset chart updates, `filters: []` clears both user filters and the
+generated dashboard-time binding. `temporal_column: null` clears only that
+generated binding, preserving user filters. Omitting those controls preserves
+the saved binding. Bullet Vega previews keep separate indexed rows even when
+dimension display labels are identical, and support both `SMART_NUMBER` and
+`SMART_NUMBER_SIGNED` number formats. Bullet preview numeric format
+precision is limited to 20 digits before formatting; raw data reads do not
+validate presentation formats.
+
+Native Bullet temporal filters retain the active filter's subject and range as a
+pair; `No filter` placeholders do not supply the subject of another active range.
+Typed configs support one such pair. Multiple active native temporal filters, or
+conflicts with explicit `temporal_column`/`time_range`, are rejected rather than
+silently dropping or moving a predicate.
+
+### Query result limits
+
+MCP chart tools accept up to 50,000 rows per query. The row-shaped `indexnames`
Review Comment:
This section lists the 50,000-row, 64 KiB cell and 16 MiB limits but not the new aggregate budgets enforced in `query_result.py` (100,000 total rows and 2,500,000 values/containers). A 50,000-row Table export with ~50 numeric columns fits every documented limit yet fails with `MalformedQueryResult`. Can these budgets and how they are counted be documented here and in the `UPDATING.md` note?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1177,372 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
Review Comment:
`_bullet_numeric_tokens` has no callers; Bullet rendering uses `_bullet_numeric_control_tokens`. Leaving the older parser means two divergent token grammars, and a later fix to this one would not change any preview. Can it be removed (along with the unused `_canonical_result_field` at line 449)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5628583743
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5729182291
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5878802856
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4156491372
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1090,342 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = float(token)
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ category_field = _unique_bullet_category_field(model.rows)
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field,)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[category_field] = (
Review Comment:
Fixed in 475698ba5a831ad8af0d6d8f427449afe8abe81e by separating the indexed row key from the category display text. Regression tests cover null/string, boolean/string, and numeric/string label collisions.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -496,6 +1090,342 @@ def _is_nan(value: Any) -> bool:
return False
+def _bullet_numeric_tokens(value: Any) -> list[float]:
+ """Parse native comma-separated Bullet threshold controls."""
+ if isinstance(value, str):
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif isinstance(value, list):
+ tokens = value
+ else:
+ return []
+ result: list[float] = []
+ for token in tokens:
+ try:
+ number = float(token)
+ except (TypeError, ValueError):
+ continue
+ if not _is_nan(number) and math.isfinite(number):
+ result.append(number)
+ return result
+
+
+def _bullet_numeric_control_tokens(value: Any, role: str) -> list[float]: # noqa: C901
+ """Drop non-numeric native tokens like Explore, retaining safety bounds."""
+ value = _safe_enum_backing(value)
+ if value is None or (type(value) is str and value == ""):
+ return []
+ if type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role} exceeds the size limit")
+ tokens: list[Any] = [token.strip() for token in value.split(",")]
+ elif type(value) is list:
+ tokens = [
+ list.__getitem__(value, index) for index in range(list.__len__(value))
+ ]
+ else:
+ raise BulletOutputError(f"Bullet {role} must be a comma-separated list")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError(f"Bullet {role} exceeds the item limit")
+
+ numbers: list[float] = []
+ for index, token in enumerate(tokens):
+ token = _safe_enum_backing(token)
+ if type(token) is str and token == "":
+ continue
+ if type(token) is bool or not (
+ type(token) is str
+ or type(token) is int
+ or type(token) is float
+ or type(token) is Decimal
+ ):
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ if type(token) is str and len(token) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric")
+ try:
+ number = float(token)
+ except ValueError:
+ # Native controls tolerate stray text and incomplete input.
+ continue
+ except (TypeError, OverflowError) as ex:
+ raise BulletOutputError(f"Bullet {role}[{index}] is not numeric") from ex
+ if math.isnan(number):
+ continue
+ if not math.isfinite(number):
+ raise BulletOutputError(f"Bullet {role}[{index}] is NaN or infinite")
+ numbers.append(number)
+ return numbers
+
+
+def _generate_bullet_vega_lite_preview( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> VegaLitePreview:
+ """Build a horizontal layered preview from the shared strict model."""
+ model = resolve_bullet_render_model(data, form_data)
+
+ category_field = _unique_bullet_category_field(model.rows)
+ containing_range_labels = (
+ [
+ _containing_bullet_range_label(measure, model.ranges, model.range_labels)
+ for measure in model.measures
+ ]
+ if model.range_labels
+ else [None] * len(model.rows)
+ )
+ range_tooltip_field = (
+ _unique_bullet_derived_field(
+ model.rows, "__mcp_bullet_range", (category_field,)
+ )
+ if any(label is not None for label in containing_range_labels)
+ else None
+ )
+ values = []
+ for row_index, row in enumerate(model.rows):
+ copied = dict.copy(row)
+ copied[category_field] = (
+ ", ".join(
+ _bullet_category_value(dict.get(row, field), field, row_index)[1]
+ for field in model.dimensions
+ )
+ if model.dimensions
+ else ""
+ )
+ if (
+ range_tooltip_field is not None
+ and containing_range_labels[row_index] is not None
+ ):
+ copied[range_tooltip_field] = containing_range_labels[row_index]
+ values.append(copied)
+
+ y_encoding = {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ "sort": None,
+ }
+ tooltip = [
+ {
+ "field": category_field,
+ "type": "nominal",
+ "title": ", ".join(model.dimensions) if model.dimensions else None,
+ },
+ {
+ "field": model.metric_field,
+ "type": "quantitative",
+ "format": (
+ "~s" if model.y_axis_format == "SMART_NUMBER" else model.y_axis_format
+ ),
+ },
+ ]
+ if range_tooltip_field is not None:
+ tooltip.append(
+ {"field": range_tooltip_field, "type": "nominal", "title": "Range"}
+ )
+ axis_min = min(
+ 0.0,
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ axis_max = max(
+ *model.measures,
+ *model.ranges,
+ *model.markers,
+ *model.marker_lines,
+ )
+ if axis_min == axis_max:
+ axis_max = axis_min + (abs(axis_min) or 1)
+ vega_format = "~s" if model.y_axis_format == "SMART_NUMBER" else model.y_axis_format
Review Comment:
Fixed in 475698ba5a831ad8af0d6d8f427449afe8abe81e by translating SMART_NUMBER_SIGNED to +~s for both the Vega axis and tooltip. The regression test checks both format fields.
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,445 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels are optional; missing marker "
+ "labels fall back to formatted values."
+ ),
+ suggestions=[
+ "Use metric with aggregate, saved_metric, or sql_expression + label",
+ "Use dimensions (alias: groupby) for row hierarchy",
+ "Use ranges, markers, and marker_lines for comparison targets",
+ ],
+ error_code="BULLET_VALIDATION_ERROR",
+ )
+
+ def normalize_query_result(self, result: Any, form_data: Mapping[str, Any]) -> Any:
+ """Reject results that cannot size a Bullet chart without guessing."""
+ from superset.mcp_service.chart.query_result import query_result_data
+
+ data, failure = query_result_data(result, temporal_json_numbers=True)
+ if failure is not None:
+ return failure
+ rows = data[0] if data else []
+ if (error := _render_model_error(rows, form_data)) is not None:
+ return error
+ return result
+
+ def sanitize_data_rows(
+ self, data: list[Any], form_data: Mapping[str, Any]
+ ) -> tuple[list[Any], ChartError | None]:
+ """Expose rows through the same strict model the renderers use."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ model = resolve_bullet_render_model(data, dict(form_data))
+ except BulletOutputError as ex:
+ return [], ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+ # The strict model retains exact result keys while replacing unselected
+ # values with None and normalizing the selected roles. Its zero-valued
+ # ungrouped row is a render-only frontend fallback, not source query
+ # data, so an empty query exposes no rows to get-data and exports.
+ return (model.rows if data else []), None
+
+ def ascii_preview(
+ self, data: list[Any], form_data: dict[str, Any], width: int
+ ) -> str | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_bullet_chart,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_ascii_bullet_chart(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def vega_lite_preview(
+ self, data: list[Any], form_data: dict[str, Any]
+ ) -> VegaLitePreview | ChartError | None:
+ from superset.mcp_service.chart.preview_utils import (
+ _generate_bullet_vega_lite_preview,
+ BulletOutputError,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ return _generate_bullet_vega_lite_preview(data, form_data)
+ except BulletOutputError as ex:
+ return ChartError(
+ error=safe_exception_message(ex), error_type=ex.error_type
+ )
+
+ def resolve_update_config(
+ self,
+ config: Any,
+ existing_form_data: dict[str, Any],
+ *,
+ dataset_rebind: bool,
+ ) -> Any:
+ """Resolve sort targets against the saved hierarchy before mapping."""
+ if not isinstance(config, BulletChartConfig) or config.dimensions is not None:
+ return config
+ if not config.order_by:
+ return config
+ dimensions = (
+ existing_form_data.get("groupby") or []
+ if not dataset_rebind and existing_form_data.get("viz_type") == "bullet"
+ else []
+ )
+ return BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": dimensions}
+ )
+
+ def merge_update_form_data(
+ self,
+ existing_form_data: dict[str, Any],
+ new_form_data: dict[str, Any],
+ config: Any,
+ *,
+ dataset_rebind: bool,
+ ) -> dict[str, Any] | None:
+ """Merge filter provenance and preserve omitted native Bullet controls."""
+ if not isinstance(config, BulletChartConfig) or dataset_rebind:
+ return None
Review Comment:
Fixed in 475698ba5a831ad8af0d6d8f427449afe8abe81e by checking inherited query references against the replacement dataset before the cached-preview merge. Compatible omitted Bullet dimensions and dimension-based sorting are preserved, matching saved updates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4156493234
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1781,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+ """
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
Review Comment:
Fixed in 475698ba5a831ad8af0d6d8f427449afe8abe81e by discarding temporal provenance with incompatible inherited filters. Table and Bullet regressions cover replacement datasets with and without a different time column.
##########
superset/mcp_service/chart/tool/update_chart_preview.py:
##########
@@ -243,62 +294,75 @@ def update_chart_preview( # noqa: C901
if previous_form_data:
merge_table_column_config(previous_form_data, new_form_data)
merge_interactive_pivot_ui_config(previous_form_data, new_form_data)
- new_form_data = merge_chart_form_data(
- previous_form_data,
- new_form_data,
- config,
- dataset_rebind=dataset_rebind,
- )
+ merge_plugin = plugin_for_viz_type(new_form_data.get("viz_type"))
+ if merge_plugin is not None and merge_plugin.owns_update_merge:
+ # The plugin owns its temporal/rebind merge contract.
+ # Generic merges would restore deliberately removed state.
+ new_form_data = merge_chart_form_data(
+ previous_form_data,
+ new_form_data,
+ config,
+ dataset_rebind=dataset_rebind,
+ )
+ else:
+ new_form_data = merge_chart_form_data(
+ previous_form_data,
+ new_form_data,
+ config,
+ dataset_rebind=dataset_rebind,
+ )
+ merge_update_form_data(previous_form_data, new_form_data, config)
+ merge_same_viz_form_data(previous_form_data, new_form_data, config)
Review Comment:
Fixed in 475698ba5a831ad8af0d6d8f427449afe8abe81e by pruning incompatible cached query roles before the preservation pass and binding the returned datasource to the replacement dataset. Regression coverage checks grouping, sorting, filters, and stale datasource identifiers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4180329043
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,733 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Native predicates and ordering aliases are normalized before merging; explicit clears remain authoritative. Regression coverage is included.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1739,551 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Table scalar columns and metrics now use scalar-to-list normalization, including raw columns. Regression coverage is included.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -589,23 +591,518 @@ def resolve_big_number_columns(form_data: dict[str, Any]) -> list[Any]:
return [granularity] if isinstance(granularity, str) and granularity else []
-def resolve_gantt_query_fields( # noqa: C901
- form_data: dict[str, Any],
-) -> tuple[list[Any], list[Any], list[list[Any]], list[Any]]:
- """Mirror the ECharts Gantt ``buildQuery`` field extraction contract.
+def extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
+ """Return the x_axis column name from form_data, or None if not set."""
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis:
+ return x_axis
+ if isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ return col_name if isinstance(col_name, str) and col_name else None
+ return None
+
- Returns ``(columns, metrics, orderby, series_columns)``. Saved form data is
- user-editable, so malformed or oversized native ordering is rejected rather
- than silently dropped or passed into ``QueryContextFactory``.
+def _x_axis_query_field(form_data: dict[str, Any]) -> Any | None:
+ """Resolve a frontend x-axis value without losing SQL expressions."""
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis:
+ return x_axis
+ if isinstance(x_axis, dict):
+ if (
+ isinstance(x_axis.get("sqlExpression"), str)
+ and x_axis.get("sqlExpression")
+ and isinstance(x_axis.get("label"), str)
+ and x_axis.get("label")
+ and x_axis.get("expressionType") in (None, "SQL")
+ ):
+ return x_axis
+ column_name = x_axis.get("column_name") or x_axis.get("columnName")
+ if isinstance(column_name, str) and column_name:
+ return column_name
+ return None
+
+
+def _normalized_x_axis_query_field(form_data: dict[str, Any]) -> Any | None:
+ """Mirror ``buildQueryContext.normalizeTimeColumn`` for a set x-axis."""
+ x_axis = _x_axis_query_field(form_data)
+ if x_axis is None:
+ return None
+ time_grain = form_data.get("time_grain_sqla")
+ if isinstance(x_axis, str):
+ normalized = {
+ "columnType": "BASE_AXIS",
+ "sqlExpression": x_axis,
+ "label": x_axis,
+ "expressionType": "SQL",
+ "isColumnReference": True,
+ }
+ if time_grain is not None:
+ normalized["timeGrain"] = time_grain
+ return normalized
+ normalized = {"columnType": "BASE_AXIS", **x_axis}
+ # The original adhoc column's grain overrides the common control, matching
+ # the frontend spread order.
+ if "timeGrain" not in normalized and time_grain is not None:
+ normalized["timeGrain"] = time_grain
+ return normalized
+
+
+def _resolve_big_number_query_columns(form_data: dict[str, Any]) -> list[Any]:
+ """Resolve only Big Number's explicit x-axis query column.
+
+ The frontend keeps ``granularity_sqla`` out of ``columns`` and asks the
+ backend for a timeseries instead, which yields ``__timestamp``. An explicit
+ ``x_axis`` is different: the plugin retains that column in the final query.
"""
- from superset.utils import json as utils_json
+ if (x_axis := _normalized_x_axis_query_field(form_data)) is not None:
+ return [x_axis]
+ return []
+
+
+def _as_list(value: Any) -> list[Any]:
+ """Match the frontend's ``ensureIsArray`` for query controls."""
+ if value is None:
+ return []
+ return value if isinstance(value, list) else [value]
+
+
+def _column_label(column: Any) -> str | None:
+ """Return the frontend ``getColumnLabel`` value for a query column."""
+ if isinstance(column, str):
+ return column
+ if not isinstance(column, dict):
+ return None
+ return (
+ column.get("label") or column.get("sqlExpression") or column.get("column_name")
+ )
+
+
+def _metric_label(metric: Any) -> str | None:
+ """Return the frontend ``getMetricLabel`` value for a query metric."""
+ if isinstance(metric, str):
+ return metric
+ if not isinstance(metric, dict):
+ return None
+ if label := metric.get("label"):
+ return label
+ if metric.get("expressionType") == "SIMPLE":
+ column = metric.get("column") or {}
+ name = (
+ column.get("columnName") or column.get("column_name")
+ if isinstance(column, dict)
+ else None
+ )
+ if name and metric.get("aggregate"):
+ return f"{metric['aggregate']}({name})"
+ return metric.get("sqlExpression")
+
+
+def _is_query_form_metric(value: Any) -> bool:
+ """Mirror the frontend's ``isQueryFormMetric`` type guard."""
+ return isinstance(value, str) or (
+ isinstance(value, dict) and value.get("expressionType") in {"SIMPLE", "SQL"}
+ )
+
+
+def _timeseries_base_metrics(form_data: dict[str, Any]) -> list[Any]:
+ """Return metrics extracted by the common frontend query-field aliases."""
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Timeseries query reconstruction retains the singular metric alias and deduplicates metric outputs. Regression coverage is included.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -809,10 +1739,551 @@ def build_mixed_timeseries_secondary(
return qd
-# Deck.gl viz types that conditionally set is_timeseries from time_grain_sqla
-_DECK_TIMESERIES_VIZ_TYPES: frozenset[str] = frozenset(
- {"deck_arc", "deck_path", "deck_polygon", "deck_scatter", "deck_screengrid"}
-)
+def build_histogram_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Histogram buildQuery, including its histogram post-processing."""
+ column = form_data.get("column")
+ histogram_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [*histogram_groupby, column] if column is not None else histogram_groupby,
+ [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ having_filter = bool(form_data.get("having")) or any(
+ isinstance(filter_, dict) and filter_.get("clause") == "HAVING"
+ for filter_ in form_data.get("adhoc_filters") or []
+ )
+ if having_filter:
+ query["metrics"] = [
+ {
+ "expressionType": "SQL",
+ "sqlExpression": "COUNT(*)",
+ "label": "COUNT(*)",
+ }
+ ]
+ bins = form_data.get("bins", 5)
+ try:
+ parsed_bins = float(bins)
+ parsed_bins = int(parsed_bins) if parsed_bins.is_integer() else parsed_bins
+ except (TypeError, ValueError):
+ parsed_bins = 5
+ query["post_processing"] = [
+ {
+ "operation": "histogram",
+ "options": {
+ "column": _column_label(column),
+ "groupby": [
+ label
+ for item in histogram_groupby
+ if (label := _column_label(item))
+ ],
+ "bins": parsed_bins,
+ "cumulative": bool(form_data.get("cumulative")),
+ "normalize": bool(form_data.get("normalize")),
+ },
+ }
+ ]
+ return [query]
+
+
+def build_box_plot_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Box Plot buildQuery, including its boxplot post-processing."""
+ distribute = _as_list(form_data.get("columns"))
+ if not distribute and form_data.get("granularity_sqla"):
+ distribute = [form_data["granularity_sqla"]]
+ box_groupby = _as_list(form_data.get("groupby"))
+ query = build_single_query_dict(
+ form_data,
+ [
+ *(_temporal_column(column, form_data) for column in distribute),
+ *box_groupby,
+ ],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ query["series_columns"] = box_groupby
+ if whisker := form_data.get("whiskerOptions"):
+ whisker_type = "tukey"
+ percentiles: list[int] | None = None
+ if whisker == "Min/max (no outliers)":
+ whisker_type = "min/max"
+ elif match := re.fullmatch(r"(\d{1,3})/(\d{1,3}) percentiles", str(whisker)):
+ whisker_type = "percentile"
+ percentiles = [int(match.group(1)), int(match.group(2))]
+ elif whisker != "Tukey":
+ raise ValueError(f"Unsupported whisker type: {whisker}")
+ query["post_processing"] = [
+ {
+ "operation": "boxplot",
+ "options": {
+ "whisker_type": whisker_type,
+ "percentiles": percentiles,
+ "groupby": [
+ label
+ for column in box_groupby
+ if (label := _column_label(column))
+ ],
+ "metrics": [
+ label
+ for metric in query["metrics"]
+ if (label := _metric_label(metric))
+ ],
+ },
+ }
+ ]
+ return [query]
+
+
+def build_pivot_table_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pivot Table buildQuery, including subtotal grouping sets."""
+ rows = _as_list(form_data.get("groupbyRows"))
+ pivot_columns = _as_list(form_data.get("groupbyColumns"))
+ if form_data.get("transposePivot"):
+ rows, pivot_columns = pivot_columns, rows
+ columns = _dedupe_query_fields([*rows, *pivot_columns], _column_label)
+ query = build_single_query_dict(
+ form_data,
+ [_temporal_column(column, form_data) for column in columns],
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ )
+ sort_metric = query.get("series_limit_metric")
+ if sort_metric is None and query["metrics"]:
+ sort_metric = query["metrics"][0]
+ if sort_metric is not None:
+ query["orderby"] = [[sort_metric, not query.get("order_desc", True)]]
+ if grouping_sets := _pivot_grouping_sets(form_data, rows, pivot_columns):
+ query["grouping_sets"] = grouping_sets
+ return [query]
+
+
+def build_pie_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ contribution: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Pie/Sunburst buildQuery; Pie adds a contribution operator."""
+ metric = form_data.get("metric")
+ query = build_single_query_dict(
+ form_data,
+ _as_list(form_data.get("groupby")),
+ [metric] if metric is not None else [],
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ if form_data.get("sort_by_metric") and metric is not None:
+ query["orderby"] = [[metric, False]]
+ if contribution and (label := _metric_label(metric)):
+ query["post_processing"] = [
+ {
+ "operation": "contribution",
+ "options": {
+ "columns": [label],
+ "rename_columns": [f"{label}__contribution"],
+ },
+ }
+ ]
+ return [query]
+
+
+def _positive_int(value: Any) -> int:
+ """Coerce a stored limit (int, numeric string, or empty) to a positive int or 0."""
+ try:
+ coerced = int(value)
+ except (TypeError, ValueError):
+ return 0
+ return coerced if coerced > 0 else 0
+
+
+def build_table_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Table buildQuery: percent metrics, comparisons, totals, paging."""
+ raw_mode = form_data.get("query_mode") == "raw" or (
+ form_data.get("query_mode") not in {"raw", "aggregate"}
+ and bool(form_data.get("all_columns"))
+ )
+ table_columns = list(
+ (form_data.get("all_columns") or [])
+ if raw_mode
+ else (form_data.get("groupby") or [])
+ )
+ table_metrics = [] if raw_mode else list(form_data.get("metrics") or [])
+ percent_metrics = [] if raw_mode else _as_list(form_data.get("percent_metrics"))
+ table_metrics = _dedupe_query_fields(
+ [*table_metrics, *percent_metrics], _metric_label
+ )
+ table_orderby = _parse_orderby(form_data.get("order_by_cols"))
+ if not raw_mode:
+ sort_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if sort_metrics:
+ table_orderby = [[sort_metrics[0], not form_data.get("order_desc", False)]]
+ elif table_metrics:
+ table_orderby = [[table_metrics[0], False]]
+ query = build_single_query_dict(
+ form_data,
+ table_columns,
+ table_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=table_orderby,
+ )
+ if not raw_mode:
+ # Table selects one temporal axis and places it before the other roles.
+ for index, column in enumerate(table_columns):
+ temporal_column = _temporal_column(column, form_data)
+ if temporal_column is not column:
+ query["columns"] = [
+ temporal_column,
+ *table_columns[:index],
+ *table_columns[index + 1 :],
+ ]
+ break
+ offsets = _table_time_offsets(form_data, query)
+ query["time_offsets"] = offsets
+ post_processing: list[dict[str, Any]] = []
+ contribution: dict[str, Any] | None = None
+ if percent_metrics:
+ labels: list[str] = []
+ for metric in percent_metrics:
+ if label := _metric_label(metric):
+ candidates = [label]
+ if offsets:
+ candidates.extend(f"{label}__{offset}" for offset in offsets)
+ for candidate in candidates:
+ if candidate not in labels:
+ labels.append(candidate)
+ contribution = {
+ "operation": "contribution",
+ "options": {
+ "columns": labels,
+ "rename_columns": [f"%{label}" for label in labels],
+ },
+ }
+ post_processing.append(contribution)
+ if offsets and form_data.get("comparison_type") != "values":
+ source: list[str] = []
+ shifted: list[str] = []
+ for metric in table_metrics:
+ if label := _metric_label(metric):
+ for offset in offsets:
+ source.append(label)
+ shifted.append(f"{label}__{offset}")
+ post_processing.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": source,
+ "compare_columns": shifted,
+ "compare_type": form_data.get("comparison_type"),
+ "drop_original_columns": True,
+ },
+ }
+ )
+ query["post_processing"] = post_processing
+
+ # ``query["row_limit"]`` is the normalized caller limit (explicit request
+ # limit or the saved row_limit, which may be stored as a string); page
+ # sizing narrows it but never replaces it.
+ configured_limit = _positive_int(query.get("row_limit"))
+ if form_data.get("server_pagination"):
+ if page_size := _positive_int(form_data.get("server_page_length")):
+ query["row_limit"] = (
+ min(page_size, configured_limit) if configured_limit else page_size
+ )
+ query["row_offset"] = 0
+
+ extra_queries: list[dict[str, Any]] = []
+ if form_data.get("percent_metric_calculation") == "all_records" and percent_metrics:
+ extra_queries.append(
+ {
+ **query,
+ "columns": [],
+ "metrics": percent_metrics,
+ "post_processing": [],
+ "row_limit": 0,
+ "row_offset": 0,
+ "orderby": [],
+ "is_timeseries": False,
+ }
+ )
+ if table_metrics and form_data.get("show_totals") and not raw_mode:
+ totals = {
+ **query,
+ "columns": [],
+ "metrics": _table_totals_metrics(
+ table_metrics, form_data.get("totals_aggregate")
+ ),
+ "row_limit": 0,
+ "row_offset": 0,
+ "post_processing": [contribution] if contribution else [],
+ }
+ totals.pop("orderby", None)
+ totals.pop("order_desc", None)
+ extra_queries.append(totals)
+ if form_data.get("server_pagination"):
+ rowcount = {
+ **query,
+ "time_offsets": [],
+ "row_limit": configured_limit or 0,
+ "row_offset": 0,
+ "post_processing": [],
+ "is_rowcount": True,
+ }
+ return [query, rowcount, *extra_queries]
+ return [query, *extra_queries]
+
+
+def build_gantt_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Gantt buildQuery with its interval columns and series."""
+ (
+ gantt_columns,
+ gantt_metrics,
+ gantt_orderby,
+ gantt_groupby,
+ ) = resolve_gantt_query_fields(form_data)
+ query = build_single_query_dict(
+ form_data,
+ gantt_columns,
+ gantt_metrics,
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=gantt_orderby,
+ )
+ query["series_columns"] = gantt_groupby
+ return [query]
+
+
+def build_interactive_pivot_query_dicts(
+ form_data: dict[str, Any],
+ *,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Interactive Pivot Table buildQuery."""
+ interactive_columns = [
+ _temporal_column(column, form_data)
+ for column in _as_list(form_data.get("groupby"))
+ ]
+ query = build_single_query_dict(
+ form_data,
+ interactive_columns,
+ list(form_data.get("metrics") or []),
+ row_limit=row_limit,
+ order_desc=order_desc,
+ orderby=form_data.get("orderby"),
+ )
+ _normalize_orderby(query)
+ return [query]
+
+
+def build_big_number_query_dicts( # noqa: C901
+ form_data: dict[str, Any],
+ *,
+ trendline: bool,
+ engine: str,
+ row_limit: int | None,
+ order_desc: bool | None,
+) -> list[dict[str, Any]]:
+ """Render Big Number (with or without trendline) buildQuery."""
+ metric = form_data.get("metric")
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Big Number query reconstruction retains the plural-metric fallback. Regression coverage is included.
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,450 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ # Match QueryObject's guarded legacy saved-metric normalization.
+ if not ({"sqlExpression", "aggregate", "column"} & value.keys()):
+ label = value.get("label")
+ if isinstance(label, str) and label:
+ return "saved_metric", label
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
+ viz_type = form_data.get("viz_type")
+ if strict_all_form_refs and (
+ viz_type == "mixed_timeseries"
+ or (
+ isinstance(viz_type, str)
+ and (
+ viz_type.startswith("echarts_timeseries") or viz_type == "echarts_area"
+ )
+ )
+ ):
+ if (raw_x_axis := form_data.get("x_axis")) is not None and (
+ error := column_error(raw_x_axis, "form-data x_axis column")
+ ):
+ return error
+ metric_fields = [
+ "metrics",
+ "size",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ ]
+ if viz_type == "mixed_timeseries":
+ metric_fields.extend(
+ [
+ "metrics_b",
+ "size_b",
+ "timeseries_limit_metric_b",
+ "series_limit_metric_b",
+ ]
+ )
+ for field_name in metric_fields:
+ raw_value = form_data.get(field_name)
+ values = raw_value if isinstance(raw_value, list) else [raw_value]
+ for value in values:
+ if value is not None and (
+ error := metric_error(value, f"form-data {field_name} metric")
+ ):
+ return error
+
+ layer_suffixes = ("", "_b") if viz_type == "mixed_timeseries" else ("",)
+ for suffix in layer_suffixes:
+ sort_field = f"x_axis_sort{suffix}"
+ if sort_field not in form_data or form_data.get(sort_field) is None:
+ continue
+ x_axis = form_data.get(f"x_axis{suffix}", form_data.get("x_axis"))
+ allowed_labels: set[str] = set()
+ if x_axis_label := _native_column_label(x_axis):
+ allowed_labels.add(x_axis_label)
+ raw_metrics = form_data.get(f"metrics{suffix}")
+ for metric in raw_metrics if isinstance(raw_metrics, list) else []:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ raw_limit_metric = form_data.get(f"timeseries_limit_metric{suffix}")
+ limit_metrics = (
+ raw_limit_metric
+ if isinstance(raw_limit_metric, list)
+ else [raw_limit_metric]
+ )
+ for metric in limit_metrics:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ sort_value = form_data[sort_field]
+ if not isinstance(sort_value, str) or sort_value not in allowed_labels:
+ return _native_validation_error(sort_field, repr(sort_value)[:200])
+
+ if (
+ strict_all_form_refs
+ and isinstance(viz_type, str)
+ and viz_type.startswith("deck_")
+ ):
+ # Deck layers store most query roles outside common columns/metrics.
+ # Validate every renderer-consumed raw control as well as the generated
+ # QueryObject so a dataset-only rebind cannot hide or discard a stale
+ # tooltip, cross-filter, spatial, path, or metric reference.
+ for spatial_field in ("spatial", "start_spatial", "end_spatial"):
+ spatial = form_data.get(spatial_field)
+ if spatial is None:
+ continue
+ if not isinstance(spatial, dict):
+ return _native_validation_error(
+ f"form-data {spatial_field}", repr(spatial)[:200]
+ )
+ spatial_type = spatial.get("type")
+ if not isinstance(spatial_type, str):
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ role_fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if role_fields is None:
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ for role_field in role_fields:
+ spatial_value = spatial.get(role_field)
+ if spatial_value is None:
+ return _native_validation_error(
+ f"form-data {spatial_field}.{role_field} column", "missing"
+ )
+ if error := column_error(
+ spatial_value, f"form-data {spatial_field}.{role_field} column"
+ ):
+ return error
+
+ for field_name in (
+ "line_column",
+ "geojson",
+ "dimension",
+ "cross_filter_column",
+ ):
+ column_value = form_data.get(field_name)
+ if column_value is not None and (
+ error := column_error(column_value, f"form-data {field_name} column")
+ ):
+ return error
+
+ tooltip_contents = form_data.get("tooltip_contents")
+ if tooltip_contents is not None and not isinstance(tooltip_contents, list):
+ return _native_validation_error(
+ "form-data tooltip_contents", repr(tooltip_contents)[:200]
+ )
+ for index, item in enumerate(tooltip_contents or []):
+ tooltip_value: Any = None
+ if isinstance(item, str):
+ tooltip_value = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ tooltip_value = item.get("column_name")
+ if tooltip_value is not None and (
+ error := column_error(
+ tooltip_value, f"form-data tooltip_contents[{index}] column"
+ )
+ ):
+ return error
+
+ metric_values: list[tuple[str, Any]] = []
+ if viz_type not in {"deck_geojson", "deck_polygon"}:
+ for field_name in ("metrics", "metric", "size"):
+ raw_deck_metrics = form_data.get(field_name)
+ deck_metrics = (
+ raw_deck_metrics
+ if isinstance(raw_deck_metrics, list)
+ else [raw_deck_metrics]
+ )
+ metric_values.extend(
+ (f"form-data {field_name} metric", deck_metric)
+ for deck_metric in deck_metrics
+ if deck_metric is not None
+ )
+ if viz_type == "deck_polygon" and form_data.get("metric") is not None:
+ metric_values.append(("form-data metric metric", form_data.get("metric")))
+ fixed_metric_fields = (
+ ("point_radius_fixed",)
+ if viz_type in {"deck_scatter", "deck_polygon"}
+ else ()
+ ) + (("line_width",) if viz_type == "deck_path" else ())
+ for field_name in fixed_metric_fields:
+ fixed_value = form_data.get(field_name)
+ deck_metric: Any = (
+ fixed_value
+ if (
+ isinstance(fixed_value, str)
+ and fixed_value
+ and viz_type != "deck_polygon"
+ )
+ else None
+ )
+ if isinstance(fixed_value, dict) and fixed_value.get("type") == "metric":
+ deck_metric = fixed_value.get("value")
+ if deck_metric is not None:
+ metric_values.append((f"form-data {field_name} metric", deck_metric))
+ if viz_type == "deck_path" and form_data.get("breakpoint_metric") is not None:
+ metric_values.append(
+ (
+ "form-data breakpoint_metric metric",
+ form_data.get("breakpoint_metric"),
+ )
+ )
+ for role, deck_metric in metric_values:
+ if error := metric_error(deck_metric, role):
+ return error
+
+ for filter_ in form_data.get("adhoc_filters") or []:
+ if not isinstance(filter_, dict) or filter_.get("expressionType") != "SIMPLE":
+ continue
+ if not strict_all_form_refs and _is_inert_adhoc_filter(filter_):
+ continue
+ subject = filter_.get("subject")
+ clause = str(filter_.get("clause") or "WHERE").upper()
+ if clause == "HAVING" and isinstance(subject, str):
+ metric_matches = (
+ [subject]
+ if subject in saved_metrics
+ else [
+ name
+ for name in saved_metrics
+ if name.casefold() == subject.casefold()
+ ]
+ )
+ if len(metric_matches) == 1:
+ continue
+ if subject is not None and (
+ error := column_error(subject, "form-data filter column")
+ ):
+ return error
+ if filter_.get("operator") == "TEMPORAL_RANGE" and isinstance(subject, str):
+ try:
+ temporal = resolve_dataset_column(subject, dataset_context)
+ except ValueError:
+ temporal = None
+ if temporal is not None and not temporal.get("is_temporal", False):
+ return _native_validation_error("temporal filter column", subject)
+
+ # temporal_columns_lookup describes the entire datasource, not selected
+ # roles. The physical form/query column checks validate selected references.
+
+ for query_index, query in enumerate(queries, 1):
+ metric_labels: set[str] = set()
+ for column in query.get("columns") or []:
+ if error := column_error(column, f"query {query_index} column"):
+ return error
+ for column in query.get("series_columns") or []:
+ if error := column_error(column, f"query {query_index} series column"):
+ return error
+ for column in query.get("groupby") or []:
+ if error := column_error(column, f"query {query_index} groupby column"):
+ return error
+ for level in query.get("grouping_sets") or []:
+ for column in level:
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Grouping sets resolve selected logical column labels after their source columns are validated. Regression coverage is included.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,465 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
Review Comment:
Fixed in c2d1b3f6f86e588598905382052d36927595a40d. Native SQL metrics use the SQL expression when the label is missing or null. Regression coverage is included.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4180734721
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,465 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ temporal_pairs: list[tuple[str, str]] = []
+ inert_subjects: set[str] = set()
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ if not isinstance(comparator, str) or not comparator:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs a range"
+ )
+ if comparator.casefold() == "no filter":
+ inert_subjects.add(subject)
+ else:
+ temporal_pairs.append((subject, comparator))
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ if len(temporal_pairs) > 1:
+ raise ValueError(
+ "Multiple active native temporal filters cannot be represented by "
+ "a single temporal_column/time_range pair"
+ )
+ if temporal_pairs:
+ subject, comparator = temporal_pairs[0]
+ if data.get("temporal_column") not in (None, subject) or data.get(
+ "time_range"
+ ) not in (None, "No filter", comparator):
+ raise ValueError(
+ "Native temporal filter conflicts with temporal_column/time_range"
+ )
+ data["temporal_column"] = subject
+ data["time_range"] = comparator
+ elif len(inert_subjects) == 1:
+ data.setdefault("temporal_column", next(iter(inert_subjects)))
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ from superset.mcp_service.chart.preview_utils import (
+ _bullet_numeric_control_tokens,
+ )
+
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ # Saved Explore controls are lenient; newly authored typed lists
+ # still undergo the strict List[float] validation below.
+ for control in ("ranges", "markers", "marker_lines"):
+ if control in data:
+ data[control] = _bullet_numeric_control_tokens(
+ data[control], control
+ )
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
+ {"name": item} if isinstance(item, str) else item
+ for item in data[key]
+ ]
+ for key in ("order_by", "orderby", "order_by_cols"):
+ if key in data:
+ data[key] = cls._adapt_native_order_by(data[key])
+ cls._adapt_native_filters(data)
+ return data
+
+ @field_validator("ranges", "markers", "marker_lines", mode="before")
+ @classmethod
+ def tokenize_native_numeric_lists(cls, value: Any) -> Any:
+ """Parse numeric controls without creating values for empty tokens."""
+ if value is None:
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",") if token.strip()]
+ return value
+
+ @field_validator(
+ "range_labels", "marker_labels", "marker_line_labels", mode="before"
+ )
+ @classmethod
+ def tokenize_native_label_lists(cls, value: Any) -> Any:
+ """Parse label controls while preserving positional empty tokens."""
+ if value is None or value == "":
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",")]
+ return value
+
+ @field_validator("ranges", "markers", "marker_lines")
+ @classmethod
+ def reject_non_finite_values(cls, values: List[float]) -> List[float]:
+ if any(not math.isfinite(value) for value in values):
+ raise ValueError("Bullet thresholds and markers must be finite numbers")
+ return values
+
+ @field_validator("range_labels", "marker_labels", "marker_line_labels")
+ @classmethod
+ def validate_presentation_labels(cls, labels: List[str]) -> List[str]:
+ result: list[str] = []
+ for label in labels:
+ if "," in label:
+ raise ValueError(
+ "Bullet labels cannot contain commas because the frontend "
+ "comma-separated controls have no escaping"
+ )
+ if label == "":
+ result.append("")
+ continue
+ sanitized = sanitize_user_input(
+ label, "Bullet label", max_length=200, allow_empty=True
+ )
+ if sanitized is not None:
Review Comment:
With `ranges=[10,20]` and `range_labels=[" ","goal"]`, sanitization drops the first slot, so "goal" labels threshold 10 instead of 20; marker labels shift the same way. Could sanitized-empty labels retain an empty string so their positions stay aligned?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-6008037328
Fixed the required unit-test failure in ee6c82462fbe72509a2f5a8c6e69b7a4a330d5ed, after a normal merge of master in 572fce9efa.
The isolated Decimal test passes on the original PR head (8626abb61f) and freshly fetched master (ce39d221aa), one test each. The failure reproduces after merging master: upstream 2fdf756cf1 (#44949) intentionally promotes the last SQL result to top-level rows and sets its per-statement data to null to avoid duplication. The PR-added assertion was stale against that contract. Updated only that assertion; the exact Decimal string/precision checks remain intact, and production code is unchanged.
Validation on the new head: full tests/unit_tests/mcp_service suite: 8,160 passed, 4 skipped; focused SQL tool/response tests: 73 passed; staged-file pre-commit and pre-commit across all files in the PR diff: passed, including mypy and frontend type checking. No unresolved review threads. Pushed normally; fresh CI is pending.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5975599488
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4225737664
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,853 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if isinstance(existing_viz_type, str) and existing_viz_type != new_form_data.get(
Review Comment:
Valid, fixed in 968c0a8aea187ece8381839e50925b6210b72a21. `_get_existing_form_data` now injects `chart.viz_type` when params omit it (as `Slice.form_data` does), so `merge_chart_form_data`, `merge_update_form_data` and `BulletChartPlugin.resolve_update_config` all see the same boundary. `merge_update_form_data` also treats a missing saved `viz_type` as a mismatch, matching `merge_chart_form_data`. Regression tests: `test_saved_params_without_viz_type_do_not_leak_filters_across_viz_change` (Pie with `Region = US` → Table, update and preview paths) and `test_saved_bullet_params_without_viz_type_keep_native_hierarchy`.
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,853 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ # The row limit always carries the schema default so a new chart is bounded;
+ # update merging restores the saved limit when the caller omits it.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.order_dimensions
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ dimension = dimensions[index]
+ order_target = (
+ dimension.name if isinstance(dimension, ColumnRef) else dimension
+ )
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def _normalize_native_filter_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold legacy WHERE, HAVING, and filter predicates into adhoc controls."""
+ from superset.utils.core import form_data_to_adhoc, simple_filter_to_adhoc
+
+ normalized = dict(form_data)
+ legacy_filters = [
+ form_data_to_adhoc(normalized, clause)
+ for clause in ("having", "where")
+ if normalized.get(clause)
+ ]
+ legacy_filters.extend(
+ simple_filter_to_adhoc(filter_, "where")
+ for filter_ in normalized.get("filters") or []
+ if filter_ is not None
+ )
+ if legacy_filters:
+ normalized["adhoc_filters"] = [
+ *legacy_filters,
+ *(normalized.get("adhoc_filters") or []),
+ ]
+ for key in ("where", "having", "filters"):
+ normalized.pop(key, None)
+ return normalized
+
+
+def _normalize_bullet_query_aliases(form_data: Mapping[str, Any]) -> Dict[str, Any]:
+ """Fold inherited native predicates and ordering into canonical controls."""
+ from superset.mcp_service.chart.chart_helpers import _parse_orderby
+
+ normalized = _normalize_native_filter_aliases(form_data)
+ if "groupby" in normalized and not isinstance(normalized["groupby"], list):
+ # Bullet/buildQuery and transformProps read ensureIsArray(groupby).
+ normalized["groupby"] = bullet_groupby_list(normalized["groupby"])
+ if "order_by_cols" in normalized:
+ # Native extractQueryFields concatenates both aliases in key order.
+ ordering: list[Any] = []
+ for key, value in normalized.items():
+ if key == "order_by_cols":
+ ordering.extend(_parse_orderby(value))
+ elif key == "orderby":
+ ordering.extend(value or [])
+ normalized["orderby"] = ordering
+ normalized.pop("order_by_cols", None)
+ return normalized
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ existing_form_data = _normalize_bullet_query_aliases(existing_form_data)
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ "url_params",
+ # Native query context (dashboard/native filter predicates and time
+ # overrides) that buildQueryObject applies on top of the controls.
+ "extra_form_data",
+ "extra_filters",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ preserve_orderby = (
+ "orderby" not in new_form_data and "orderby" in existing_form_data
+ )
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+ if preserve_orderby:
+ new_form_data["orderby"] = _orderby_for_final_output_roles(
+ existing_form_data, new_form_data
+ )
+
+
+def _bullet_output_labels(
+ form_data: Mapping[str, Any],
+) -> tuple[set[str], dict[str, Any]]:
+ """Return a Bullet state's dimension output labels and metric outputs."""
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ dimensions = {
+ label
+ for column in bullet_groupby_list(form_data.get("groupby"))
+ if (label := _column_label(column)) is not None
+ }
+ metrics = form_data.get("metrics") or []
+ if not isinstance(metrics, (list, tuple)):
+ metrics = [metrics]
+ metric_outputs = {
+ label: metric
+ for metric in [form_data.get("metric"), *metrics]
+ if (label := _metric_label(metric)) is not None
+ }
+ return dimensions, metric_outputs
+
+
+def bullet_groupby_list(groupby: Any) -> list[Any]:
+ """Normalize a saved Bullet hierarchy like the frontend ``ensureIsArray``."""
+ if groupby is None:
+ return []
+ return list(groupby) if isinstance(groupby, (list, tuple)) else [groupby]
+
+
+def _orderby_for_final_output_roles(
+ existing_form_data: Mapping[str, Any], new_form_data: Mapping[str, Any]
+) -> Any:
+ """Drop sorts on removed output roles and rebind inherited metric expressions.
+
+ Native ordering may also rank by a saved metric or column that is not a
+ displayed output (``get_sqla_query`` resolves it independently). Those
+ sorters never named a Bullet role, so a role change does not remove them.
+ """
+ from superset.mcp_service.chart.chart_helpers import _column_label, _metric_label
+
+ saved = existing_form_data.get("orderby")
+ if not isinstance(saved, list):
+ return saved
+ outputs, metric_outputs = _bullet_output_labels(new_form_data)
+ outputs.update(metric_outputs)
+ previous_dimensions, previous_metrics = _bullet_output_labels(existing_form_data)
+ previous_outputs = previous_dimensions | set(previous_metrics)
+ retained = []
+ for entry in saved:
+ if isinstance(entry, (list, tuple)) and entry:
+ target = entry[0]
+ label = (
+ _metric_label(target)
+ or _column_label(target)
+ or target.get("metric_name")
+ if isinstance(target, Mapping)
+ else target
+ )
+ if (
+ isinstance(label, str)
+ and label not in outputs
+ and label in previous_outputs
+ ):
+ continue
+ if (
+ isinstance(target, Mapping)
+ and isinstance(label, str)
+ and label in metric_outputs
+ ):
+ # Label equality identifies an output role, not expression
+ # equality: execute the final metric, never the saved expression.
+ entry = [metric_outputs[label], *entry[1:]]
+ retained.append(entry)
+ return retained
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+
+ State never crosses a visualization boundary: a viz-type change starts from
+ the mapper's output, so the previous chart's predicates are not restored.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if isinstance(existing_viz_type, str) and existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+ existing_form_data = _normalize_native_filter_aliases(existing_form_data)
+ # The initial overlay may carry legacy keys from saved form data. Filter
+ # omission/replacement below owns the complete canonical predicate sequence.
+ for key in ("where", "having", "filters"):
+ new_form_data.pop(key, None)
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+ if (
+ existing_binding is None
+ and incoming_binding is not None
+ and isinstance(incoming_subject, str)
+ and "filters" not in explicit_fields
+ and temporal_explicit
+ ):
+ # A saved temporal filter that Explore wrote has no MCP provenance
+ # marker. When it is the only native filter for the incoming subject,
+ # the update replaces it in place instead of appending a duplicate.
+ native_matches = [
+ filter_
+ for filter_ in existing_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == incoming_subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(native_matches) == 1:
+ existing_binding = native_matches[0]
+ existing_subject = incoming_subject
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+
+def _currency_form_value(value: CurrencyFormat | None) -> dict[str, str] | None:
+ """Return the native value for an explicitly supplied currency control."""
+ return value.to_form_data() if value is not None else None
+
+
+def _column_names(value: Sequence[ColumnRef] | None) -> list[str | None] | None:
+ """Return a native column-name list while retaining an explicit null."""
+ return [column.name for column in value] if value is not None else None
+
+
+def _table_sort_value(value: Sequence[str | SortByConfig] | None) -> list[str] | None:
+ """Return the native Table sort control for an explicit typed value."""
+ if value is None:
+ return None
+ return [
+ json.dumps(
+ [entry.column, entry.ascending]
+ if isinstance(entry, SortByConfig)
+ else [entry, False]
+ )
+ for entry in value
+ ]
+
+
+def _table_column_config_value(value: Any) -> dict[str, Any] | None:
+ """Return Table column config without losing an explicit null or empty map."""
+ if value is None:
+ return None
+ return {
+ label: column.model_dump(by_alias=True, exclude_unset=True)
+ for label, column in value.items()
+ }
+
+
+# Mappers intentionally omit optional controls so fresh charts use the frontend
+# defaults. During a same-viz update, however, an explicitly supplied false,
+# null, or empty value must block preservation of the saved native key. Keep the
+# typed-to-native relationship declarative so every update path shares it.
+_FormValueConverter = Callable[[Any], Any]
+_FormControlMap = dict[str, tuple[str, _FormValueConverter]]
+
+_COMMON_EXPLICIT_FORM_CONTROLS: _FormControlMap = {
+ "color_scheme": ("color_scheme", lambda value: value),
+ "currency_format": ("currency_format", _currency_form_value),
+ "show_value": ("show_value", lambda value: value),
+}
+
+_CHART_EXPLICIT_FORM_CONTROLS: dict[str, _FormControlMap] = {
+ "table": {
+ "sort_by": ("order_by_cols", _table_sort_value),
+ "column_config": ("column_config", _table_column_config_value),
+ },
+ "xy": {
+ "group_by": ("groupby", _column_names),
+ "series_limit": ("series_limit", lambda value: value),
+ "stacked": ("stack", lambda value: "Stack" if value else None),
+ "orientation": ("orientation", lambda value: value),
+ "legend_orientation": ("legendOrientation", lambda value: value),
+ "x_axis_time_format": ("x_axis_time_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "mixed_timeseries": {
+ "group_by": ("groupby", _column_names),
+ "group_by_secondary": ("groupby_b", _column_names),
+ "currency_format_secondary": (
+ "currency_format_secondary",
+ _currency_form_value,
+ ),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "waterfall": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "big_number": {
+ "subheader": ("subheader", lambda value: value),
+ "y_axis_format": ("y_axis_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "compare_lag": ("compare_lag", lambda value: value),
+ "time_format": ("time_format", lambda value: value),
+ "aggregation": ("aggregation", lambda value: value),
+ },
+ "handlebars": {
+ "style_template": ("styleTemplate", lambda value: value),
+ "columns": ("all_columns", _column_names),
+ "groupby": ("groupby", _column_names),
+ "metrics": ("metrics", _column_names),
+ },
+ "pivot_table": {
+ "date_format": ("date_format", lambda value: value),
+ },
+ "interactive_pivot": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "series_limit": ("series_limit", lambda value: value),
+ "date_format": ("date_format", lambda value: value),
+ "column_sort": ("colOrder", lambda value: value),
+ },
+}
+
+
+def _apply_explicit_form_controls( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply typed controls whose mapper omission represents a native clear."""
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ controls = {
+ **_COMMON_EXPLICIT_FORM_CONTROLS,
+ **_CHART_EXPLICIT_FORM_CONTROLS.get(config.chart_type, {}),
+ }
+ for field_name, (native_key, convert) in controls.items():
+ if field_name in explicit_fields:
+ converted = convert(getattr(config, field_name))
+ is_clear = (
+ converted is None
+ or converted is False
+ or converted == ""
+ or converted in ([], {})
+ )
+ if is_clear:
+ new_form_data[native_key] = converted
+
+ axis_controls = {
+ "xy": (
+ ("x_axis", "x_axis_title", "x_axis_format", None),
+ ("y_axis", "y_axis_title", "y_axis_format", "logAxis"),
+ ),
+ "mixed_timeseries": (
+ ("x_axis", "xAxisTitle", "x_axis_time_format", None),
+ ("y_axis", "yAxisTitle", "y_axis_format", "logAxis"),
+ (
+ "y_axis_secondary",
+ "yAxisTitleSecondary",
+ "y_axis_format_secondary",
+ "logAxisSecondary",
+ ),
+ ),
+ }
+ if config.chart_type in axis_controls:
+ for field_name, title_key, format_key, scale_key in axis_controls[
+ config.chart_type
+ ]:
+ if field_name not in explicit_fields:
+ continue
+ axis = getattr(config, field_name)
+ if axis is None:
+ new_form_data[title_key] = None
+ new_form_data[format_key] = None
+ if scale_key:
+ new_form_data[scale_key] = None
+ continue
+ axis_fields = set(axis.model_fields_set)
+ if "title" in axis_fields:
+ new_form_data[title_key] = axis.title
+ if "format" in axis_fields:
+ new_form_data[format_key] = axis.format
+ if scale_key and "scale" in axis_fields:
+ new_form_data[scale_key] = (
+ None if axis.scale is None else axis.scale == "log"
+ )
+
+ if config.chart_type == "xy" and "legend" in explicit_fields:
+ legend = config.legend
+ if legend is None:
+ new_form_data["show_legend"] = None
+ new_form_data["legendOrientation"] = None
+ else:
+ legend_fields = set(legend.model_fields_set)
+ if "show" in legend_fields:
+ new_form_data["show_legend"] = legend.show
+ if "position" in legend_fields:
+ new_form_data["legendOrientation"] = legend.position
+
+ if config.chart_type == "interactive_pivot":
+ if "temporal_column" in explicit_fields and config.temporal_column is None:
+ new_form_data["granularity_sqla"] = None
+ new_form_data["temporal_columns_lookup"] = None
+ if (
+ "series_limit_metric" in explicit_fields
+ and config.series_limit_metric is None
+ ):
+ new_form_data["series_limit_metric"] = None
+ if "comparison_period" in explicit_fields and config.comparison_period is None:
+ new_form_data["time_compare"] = None
+ if "comparison_type" in explicit_fields and config.comparison_type is None:
+ new_form_data["comparison_type"] = None
+
+ # A Waterfall axis replacement cannot inherit a bucket belonging to the old
+ # temporal subject. Grain omission preserves only while the axis is stable;
+ # explicit null is already handled by the declarative control map above.
+ if (
+ config.chart_type == "waterfall"
+ and existing_form_data.get("x_axis") != new_form_data.get("x_axis")
+ and "time_grain" not in explicit_fields
+ ):
+ new_form_data["time_grain_sqla"] = None
+
+
+def merge_same_viz_form_data(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Preserve saved controls that the typed mapper does not represent.
+
+ The typed MCP surface deliberately models a bounded subset of every Explore
+ control panel. For a replacement within the exact same native ``viz_type``,
+ keys absent from the mapper therefore represent omitted controls and retain
+ their saved values. Mapper output and the chart-specific merge helpers run
+ first and remain authoritative, including explicit empty, false, null, and
+ nested values.
+
+ No generic state crosses a visualization boundary. This prevents query-role
+ keys from the previous plugin (for example ``metric`` or ``groupby``) from
+ leaking into a different plugin whose role contract is unrelated.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if not isinstance(existing_viz_type, str) or existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+
+ _apply_explicit_form_controls(existing_form_data, new_form_data, config)
+
+ for key, value in existing_form_data.items():
+ if key in {"where", "having", "filters"}:
+ # merge_update_form_data already resolved these legacy predicates
+ # into adhoc_filters; restoring them would undo replacement/clear.
+ continue
+ if key == MCP_DASHBOARD_TIME_FILTER_SUBJECT:
+ # merge_update_form_data owns this provenance marker. Its absence
+ # may be an intentional subject clear and must not be undone by the
+ # generic preservation layer.
+ continue
+ if key not in new_form_data:
+ new_form_data[key] = value
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet controls without reinterpreting inherited query roles.
+
+ Saved Explore state may contain SQL dimensions and SQL WHERE/SIMPLE HAVING
+ filters beyond the typed authoring surface. Omitted roles are validated by
+ the native query contract and compilation, not as newly authored physical
+ columns or SIMPLE WHERE filters. Only this validation copy excludes them
+ and native query metadata; compiled and persisted form data stays intact.
+ Explicit replacements, including ``[]``, retain strict typed validation.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ for native_query_key in ("url_params", "extra_form_data", "extra_filters"):
+ validation_data.pop(native_query_key, None)
+ if update_config is None or isinstance(update_config, BulletChartConfig):
+ if update_config is None or update_config.dimensions is None:
+ validation_data.pop("groupby", None)
+ if update_config is None or "filters" not in update_config.model_fields_set:
Review Comment:
Valid, fixed in 968c0a8aea187ece8381839e50925b6210b72a21. `validate_merged_bullet_form_data` now drops a null/empty `_mcp_dashboard_time_filter_subject` from the validation copy regardless of `filters`; a non-empty marker is still validated against its binding. Regression test: `test_bullet_null_time_subject_marker_validates_with_filters`.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,120 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list, _time_comparison
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ def vega_field(name: str) -> str:
+ """Escape a literal key so Vega-Lite does not read it as a nested path."""
+ return "".join("\\" + char if char in ".[]\\" else char for char in name)
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ series_labels = list(metric_labels)
+ if len(metric_labels) == 1 and _time_comparison(
+ form_data, _as_list(form_data.get("metrics"))
+ ):
+ # A single compared metric renames each shifted series to its bare
+ # offset, so those series carry the offset rather than the metric.
+ series_labels.extend(
+ str(offset) for offset in _as_list(form_data.get("time_compare"))
+ )
+ prefixes = {
+ spelling
+ for label in series_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(prefix + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(prefix + "__")
+ for prefix in prefixes
+ )
+ ]
+ if not fields and len(metric_labels) == 1 and form_data.get("truncate_metric"):
+ # A single truncated metric drops its label from the pivoted column
+ # names, so every non-x-axis column is one category series.
+ fields = [field for field in data[0] if field != x_axis]
+ if not fields:
+ return None
+ sample = data[0][x_axis]
+ x_type = (
+ "temporal"
+ if isinstance(sample, str) and any(char in sample for char in "-/: ")
Review Comment:
Valid, fixed in 968c0a8aea187ece8381839e50925b6210b72a21. This renderer receives no column metadata, so the x type now comes from the values themselves: `_xy_pivot_x_type` returns temporal only when every non-null x value is a date/datetime or parses as an ISO date/datetime, quantitative when all are numbers, otherwise nominal. `New York` / `Product-B` stay nominal. Regression test: `test_xy_preview_x_type_requires_parseable_dates`.
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,2066 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+_EXCEL_MIN_DATE = date(1900, 1, 1)
+# Leave room for XLSX readers' millisecond rounding at the final serial day.
+_EXCEL_MAX_TIME = time(23, 59, 59, 999000)
+_EXCEL_MAX_DATETIME = datetime.combine(date.max, _EXCEL_MAX_TIME)
+# Serial day 2958465 is 9999-12-31 in Excel's 1900 date system.
+_EXCEL_MAX_DURATION = timedelta(
+ days=2958465, hours=23, minutes=59, seconds=59, milliseconds=999
+)
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
+
+
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
Review Comment:
Valid, fixed in 968c0a8aea187ece8381839e50925b6210b72a21. `_normalize_row_value` now nulls builtin and NumPy float infinities nested inside a cell (depth > 1: lists, arrays, objects) the same way NaN maps to None. A top-level injected infinity is still rejected (`test_query_result_rejects_infinity_outside_the_producer_boundary` unchanged). Regression test: `test_query_result_nulls_infinity_nested_in_a_cell` (list, `float8[]`-style ndarray, nested object).
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -589,23 +591,518 @@ def resolve_big_number_columns(form_data: dict[str, Any]) -> list[Any]:
return [granularity] if isinstance(granularity, str) and granularity else []
-def resolve_gantt_query_fields( # noqa: C901
- form_data: dict[str, Any],
-) -> tuple[list[Any], list[Any], list[list[Any]], list[Any]]:
- """Mirror the ECharts Gantt ``buildQuery`` field extraction contract.
+def extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
+ """Return the x_axis column name from form_data, or None if not set."""
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis:
+ return x_axis
+ if isinstance(x_axis, dict):
+ col_name = x_axis.get("column_name")
+ return col_name if isinstance(col_name, str) and col_name else None
+ return None
+
+
+def _x_axis_query_field(form_data: dict[str, Any]) -> Any | None:
+ """Resolve a frontend x-axis value without losing SQL expressions."""
+ x_axis = form_data.get("x_axis")
+ if isinstance(x_axis, str) and x_axis:
+ return x_axis
+ if isinstance(x_axis, dict):
+ if (
+ isinstance(x_axis.get("sqlExpression"), str)
+ and x_axis.get("sqlExpression")
+ and isinstance(x_axis.get("label"), str)
+ and x_axis.get("label")
+ and x_axis.get("expressionType") in (None, "SQL")
+ ):
+ return x_axis
+ column_name = x_axis.get("column_name") or x_axis.get("columnName")
+ if isinstance(column_name, str) and column_name:
+ return column_name
+ return None
+
+
+def _normalized_x_axis_query_field(form_data: dict[str, Any]) -> Any | None:
+ """Mirror ``buildQueryContext.normalizeTimeColumn`` for a set x-axis."""
+ x_axis = _x_axis_query_field(form_data)
+ if x_axis is None:
+ return None
+ time_grain = form_data.get("time_grain_sqla")
+ if isinstance(x_axis, str):
+ normalized = {
+ "columnType": "BASE_AXIS",
+ "sqlExpression": x_axis,
+ "label": x_axis,
+ "expressionType": "SQL",
+ "isColumnReference": True,
+ }
+ if time_grain is not None:
+ normalized["timeGrain"] = time_grain
+ return normalized
+ normalized = {"columnType": "BASE_AXIS", **x_axis}
+ # The original adhoc column's grain overrides the common control, matching
+ # the frontend spread order.
+ if "timeGrain" not in normalized and time_grain is not None:
+ normalized["timeGrain"] = time_grain
+ return normalized
+
+
+def _resolve_big_number_query_columns(form_data: dict[str, Any]) -> list[Any]:
+ """Resolve only Big Number's explicit x-axis query column.
+
+ The frontend keeps ``granularity_sqla`` out of ``columns`` and asks the
+ backend for a timeseries instead, which yields ``__timestamp``. An explicit
+ ``x_axis`` is different: the plugin retains that column in the final query.
+ """
+ if (x_axis := _normalized_x_axis_query_field(form_data)) is not None:
+ return [x_axis]
+ return []
+
+
+def _as_list(value: Any) -> list[Any]:
+ """Match the frontend's ``ensureIsArray`` for query controls."""
+ if value is None:
+ return []
+ return value if isinstance(value, list) else [value]
+
+
+def _column_label(column: Any) -> str | None:
+ """Return the frontend ``getColumnLabel`` value for a query column."""
+ if isinstance(column, str):
+ return column
+ if not isinstance(column, dict):
+ return None
+ return (
+ column.get("label") or column.get("sqlExpression") or column.get("column_name")
+ )
+
+
+def _metric_label(metric: Any) -> str | None:
+ """Return the frontend ``getMetricLabel`` value for a query metric."""
+ if isinstance(metric, str):
+ return metric
+ if not isinstance(metric, dict):
+ return None
+ if label := metric.get("label"):
+ return label
+ if metric.get("expressionType") == "SIMPLE":
+ column = metric.get("column") or {}
+ name = (
+ column.get("columnName") or column.get("column_name")
+ if isinstance(column, dict)
+ else None
+ )
+ if name and metric.get("aggregate"):
+ return f"{metric['aggregate']}({name})"
+ return metric.get("sqlExpression")
+
+
+def _is_query_form_metric(value: Any) -> bool:
+ """Mirror the frontend's ``isQueryFormMetric`` type guard."""
+ return isinstance(value, str) or (
+ isinstance(value, dict) and value.get("expressionType") in {"SIMPLE", "SQL"}
+ )
- Returns ``(columns, metrics, orderby, series_columns)``. Saved form data is
- user-editable, so malformed or oversized native ordering is rejected rather
- than silently dropped or passed into ``QueryContextFactory``.
+
+def _timeseries_base_metrics(form_data: dict[str, Any]) -> list[Any]:
+ """Return metrics extracted by the common frontend query-field aliases."""
+ metrics = [*_as_list(form_data.get("metrics")), *_as_list(form_data.get("metric"))]
+ if (size := form_data.get("size")) is not None:
+ metrics.append(size)
+ return _dedupe_query_fields(metrics, _metric_label)
+
+
+def _timeseries_extra_metrics(form_data: dict[str, Any]) -> list[Any]:
+ """Mirror ``extractExtraMetrics`` for ungrouped x-axis sorting."""
+ if _as_list(form_data.get("groupby")):
+ return []
+ limit_metrics = _as_list(form_data.get("timeseries_limit_metric"))
+ if not limit_metrics:
+ return []
+ limit_metric = limit_metrics[0]
+ limit_label = _metric_label(limit_metric)
+ if not limit_label or limit_label != form_data.get("x_axis_sort"):
+ return []
+ if any(
+ _metric_label(metric) == form_data.get("x_axis_sort")
+ for metric in _as_list(form_data.get("metrics"))
+ ):
+ return []
+ return [limit_metric]
+
+
+def _query_series_columns(query: dict[str, Any]) -> list[Any]:
+ """Resolve pivot columns with JavaScript's array-truthiness semantics.
+
+ JavaScript treats an explicitly empty array as truthy, so
+ ``series_columns: []`` must not fall through to the query's x-axis column.
+ Only an absent/null series-columns field falls back to ``columns``.
"""
- from superset.utils import json as utils_json
+ if "series_columns" in query and query["series_columns"] is not None:
+ return _as_list(query["series_columns"])
+ return _as_list(query.get("columns"))
+
+
+def _dedupe_query_fields(values: list[Any], labeler: Any) -> list[Any]:
+ """Preserve the first query field for each frontend result label."""
+ result: list[Any] = []
+ seen: set[str] = set()
+ for value in values:
+ label = labeler(value)
+ if not label or label in seen:
+ continue
+ seen.add(label)
+ result.append(value)
+ return result
+
+
+def _deck_tooltip_columns(value: Any) -> list[str]:
+ """Extract the physical tooltip fields accepted by Deck.gl plugins."""
+ if not isinstance(value, list):
+ return []
+ columns: list[str] = []
+ for item in value:
+ column: Any = None
+ if isinstance(item, str):
+ column = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ column = item.get("column_name")
+ if isinstance(column, str) and column and column not in columns:
+ columns.append(column)
+ return columns
+
+
+def _deck_base_query_fields( # noqa: C901
+ form_data: dict[str, Any],
+) -> tuple[list[Any], list[Any], list[list[Any]] | None]:
+ """Mirror the common fields built before a Deck.gl layer adapter runs."""
+ query_mode = form_data.get("query_mode")
+ columns: list[Any] = []
+ metrics: list[Any] = []
+ raw_orderby: list[Any] = []
+ aliases = {
+ "metric": "metrics",
+ "metric_2": "metrics",
+ "secondary_metric": "metrics",
+ "left_metric": "metrics",
+ "right_metric": "metrics",
+ "x": "metrics",
+ "y": "metrics",
+ "size": "metrics",
+ "all_columns": "columns",
+ "series": "groupby",
+ "order_by_cols": "orderby",
+ }
+ for key, value in form_data.items():
+ if value is None:
+ continue
+ normalized = aliases.get(key, key)
+ if query_mode == "aggregate" and normalized == "columns":
+ continue
+ if query_mode == "raw" and normalized in {"groupby", "metrics"}:
+ continue
+ if normalized == "groupby":
+ normalized = "columns"
+ if normalized == "columns":
+ columns.extend(_as_list(value))
+ elif normalized == "metrics":
+ metrics.extend(_as_list(value))
+ elif normalized == "orderby":
+ raw_orderby.extend(_as_list(value))
+
+ orderby: list[list[Any]] = []
+ if len(raw_orderby) > 100:
+ raise ValueError("Deck orderby must contain at most 100 entries")
+ for index, value in enumerate(raw_orderby):
+ if isinstance(value, str):
+ if len(value) > 1000:
+ raise ValueError(f"Deck orderby[{index}] is too long")
+ try:
+ value = utils_json.loads(value)
+ except (TypeError, ValueError) as ex:
+ raise ValueError(f"Deck orderby[{index}] is not valid JSON") from ex
+ if (
+ not isinstance(value, (list, tuple))
+ or len(value) != 2
+ or not isinstance(value[1], bool)
+ ):
+ raise ValueError(
+ f"Deck orderby[{index}] must be [field, ascending_boolean]"
+ )
+ orderby.append(list(value))
+
+ return (
+ _dedupe_query_fields(columns, _column_label),
+ _dedupe_query_fields(metrics, _metric_label),
+ orderby or None,
+ )
+
+
+def _deck_add_columns(columns: list[Any], *values: Any) -> list[Any]:
+ """Add layer fields by frontend result label without duplicates."""
+ expanded = list(columns)
+ for value in values:
+ expanded.extend(_as_list(value))
+ return _dedupe_query_fields(expanded, _column_label)
+
+
+def _deck_add_metrics(metrics: list[Any], *values: Any) -> list[Any]:
+ """Add layer metric roles without emitting the same output label twice."""
+ expanded = list(metrics)
+ for value in values:
+ expanded.extend(_as_list(value))
+ return _dedupe_query_fields(expanded, _metric_label)
+
+
+def _deck_fixed_or_metric(
+ value: Any, *, allow_legacy_string: bool = True
+) -> Any | None:
+ """Return the metric stored in a Deck fixed-or-metric control."""
+ if allow_legacy_string and isinstance(value, str) and value:
+ # Legacy Deck controls store the metric key directly as a string.
+ return value
+ if not isinstance(value, dict) or value.get("type") != "metric":
+ return None
+ metric = value.get("value")
+ return (
+ metric if metric is not None and not isinstance(metric, (int, float)) else None
+ )
+
+
+def _deck_spatial_columns(value: Any) -> list[str]:
+ """Resolve one complete frontend Deck spatial configuration."""
+ if not isinstance(value, dict) or not value.get("type"):
+ raise ValueError("Bad spatial key")
+ spatial_type = value["type"]
+ fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if fields is None:
+ raise ValueError(f"Unknown spatial type: {spatial_type}")
+ columns: list[str] = []
+ for field in fields:
+ column = value.get(field)
+ if not isinstance(column, str) or not column:
+ raise ValueError(f"Incomplete {spatial_type} spatial configuration")
+ columns.append(column)
+ return columns
+
+
+def _deck_add_null_filters(
+ query: dict[str, Any], columns: list[str], *, include_null_value: bool = False
+) -> None:
+ """Append renderer-required non-null filters, deduplicated by column."""
+ filters = list(query.get("filters") or [])
+ present = {
+ item.get("col")
+ for item in filters
+ if isinstance(item, dict) and item.get("op") == "IS NOT NULL"
+ }
+ for column in columns:
+ if not column or column in present:
+ continue
+ filter_: dict[str, Any] = {"col": column, "op": "IS NOT NULL"}
+ if include_null_value:
+ filter_["val"] = None
+ filters.append(filter_)
+ present.add(column)
+ query["filters"] = filters
+
+
+def _build_deck_query( # noqa: C901
Review Comment:
Valid, fixed in 968c0a8aea187ece8381839e50925b6210b72a21. Removed `resolve_deck_gl_columns`, `_resolve_deck_gl_metrics`, `_deck_gl_null_filters`, `_deck_gl_spatial_cols` and `_is_metric_ref` and their unit tests. The `build_query_dicts_from_form_data` Deck tests, which exercise the production `_build_deck_query` path, are kept.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3944687848
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -323,6 +370,536 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip()
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = float(Decimal(stripped))
+ except (InvalidOperation, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet number format {format_!r} is unsupported by previews",
+ error_type="UnsupportedFormat",
+ ) from ex
+ return format_
+
+
+def _format_bullet_number(format_: str, value: float) -> str:
+ """Format finite Bullet values, including the full binary-float range."""
+ from superset.utils.number_format import format_numeric
+
+ try:
+ return format_numeric(format_, value)
+ except OverflowError:
+ # SMART_NUMBER's significant-digit rounding can overflow a finite float
+ # near DBL_MAX. Scientific repr remains deterministic and informative.
+ if format_ in {"SMART_NUMBER", "SMART_NUMBER_SIGNED"} and math.isfinite(value):
+ prefix = "+" if format_ == "SMART_NUMBER_SIGNED" and value > 0 else ""
+ return prefix + repr(value)
+ raise
+
+
+def _containing_bullet_range_label(
+ measure: float, ranges: list[float], labels: list[str]
+) -> str | None:
+ """Match the frontend's labelled containing-range tooltip selection."""
+ ascending = sorted(zip(ranges, labels, strict=True), key=lambda entry: entry[0])
+ for threshold, label in ascending:
+ if measure <= threshold:
+ return label or None
+ if ascending and ascending[-1][1]:
+ return f"> {ascending[-1][1]}"
+ return None
+
+
+def resolve_bullet_render_model( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> BulletRenderModel:
+ """Resolve and validate every Bullet row and presentation control."""
+ if type(data) is not list:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ for row_index in range(list.__len__(data)):
+ row = list.__getitem__(data, row_index)
+ if type(row) is not dict:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ if dict.__len__(row) > _MAX_BULLET_FIELDS:
+ raise BulletOutputError("Bullet query row exceeds the field limit")
+ for key in dict.keys(row):
+ if type(key) is not str:
+ raise BulletOutputError("Bullet query row keys must be strings")
+ if len(key) > _MAX_BULLET_FIELD_BYTES:
+ raise BulletOutputError("Bullet query row key exceeds the size limit")
+
+ if type(form_data) is not dict:
+ raise BulletOutputError("Bullet form data must be an object")
+
+ metric_label = _form_metric_label(dict.get(form_data, "metric"))
+ if not metric_label:
+ raise BulletOutputError("Bullet metric has no declared result alias")
+ raw_groupby = dict.get(form_data, "groupby")
+ if raw_groupby is None:
+ raw_groupby = []
+ if type(raw_groupby) is not list:
+ raise BulletOutputError("Bullet dimensions must be an array")
+ dimension_labels = [
+ _form_column_label(list.__getitem__(raw_groupby, index))
+ for index in range(list.__len__(raw_groupby))
+ ]
+ if any(not label for label in dimension_labels):
+ raise BulletOutputError("Bullet dimension has no declared result alias")
+
+ if data:
+ first_row = list.__getitem__(data, 0)
+ metric_field = _require_result_field(metric_label, first_row, "metric")
+ dimensions = [
+ _require_result_field(label, first_row, "dimension")
+ for label in dimension_labels
+ ]
+ else:
+ # The frontend accepts empty results. Ungrouped charts retain one
+ # zero-valued measure; grouped charts retain the declared roles but no
+ # categories or rows are fabricated.
+ metric_field = metric_label
+ dimensions = [label for label in dimension_labels if label is not None]
+
+ measures: list[float] = []
+ copied_rows: list[dict[str, Any]] = []
+ for index in range(list.__len__(data)):
+ row = list.__getitem__(data, index)
+ row_metric_field = _require_result_field(
+ metric_label, row, f"metric row {index}"
+ )
+ measure = _bullet_number(
+ dict.__getitem__(row, row_metric_field), index, metric_field
+ )
+ # Reserve every exact output key so the internal Vega category alias
+ # cannot collide with an unselected result field. Unselected values are
+ # deliberately replaced with None rather than converted or serialized.
+ copied: dict[str, Any] = dict.fromkeys(dict.keys(row))
+ copied[metric_field] = measure
+ for label, dimension in zip(dimension_labels, dimensions, strict=True):
+ row_dimension = _require_result_field(label, row, f"dimension row {index}")
+ dimension_value, _ = _bullet_category_value(
+ dict.__getitem__(row, row_dimension), dimension, index
+ )
+ copied[dimension] = dimension_value
+ copied_rows.append(copied)
+ measures.append(measure)
+
+ # The frontend validates/coerces the whole result array but renders only
+ # the first row for an ungrouped aggregate.
+ if not dimensions:
+ copied_rows = copied_rows[:1]
+ measures = measures[:1]
+ if not copied_rows:
+ copied_rows = [{metric_field: 0.0}]
+ measures = [0.0]
+
+ ranges = _strict_bullet_numeric_tokens(dict.get(form_data, "ranges"), "ranges")
+ if not ranges:
+ # Match Bullet/transformProps.ts: the largest measure drives one
+ # qualitative band whose upper threshold is 110% of that measure.
+ ranges = [0.0, max(measures, default=0.0) * 1.1]
+ markers = _strict_bullet_numeric_tokens(dict.get(form_data, "markers"), "markers")
+ marker_lines = _strict_bullet_numeric_tokens(
+ dict.get(form_data, "marker_lines"), "marker lines"
+ )
+ all_numbers = [*measures, *ranges, *markers, *marker_lines]
+ if any(not math.isfinite(value) for value in all_numbers):
+ raise BulletOutputError("Bullet presentation values must be finite")
+
+ show_labels = dict.get(form_data, "show_labels", False)
+ show_legend = dict.get(form_data, "show_legend", False)
+ if type(show_labels) is not bool or type(show_legend) is not bool:
+ raise BulletOutputError("Bullet label and legend controls must be booleans")
+
+ range_labels = _bullet_string_tokens(dict.get(form_data, "range_labels"))
+ marker_labels = _bullet_string_tokens(dict.get(form_data, "marker_labels"))
+ marker_line_labels = _bullet_string_tokens(
+ dict.get(form_data, "marker_line_labels")
+ )
+ for role, labels, values in (
+ ("range", range_labels, ranges),
+ ("marker", marker_labels, markers),
+ ("marker line", marker_line_labels, marker_lines),
+ ):
+ if labels and len(labels) != len(values):
Review Comment:
Fixed in 1e3150c8ae8d8cb5ed1ace38de6159cf7ac2558c. Removed label/value cardinality rejection from both typed validation and the render model. Missing/empty range labels stay blank; marker and marker-line labels fall back by original index to the formatted number. The containing-range tooltip also handles short lists, and sorted bands retain their own original labels. Extra labels are ignored, matching Explore.
Evidence: test_bullet_short_labels_and_case_distinct_dimensions_reach_fastmcp exercises real saved and cached get_chart_preview calls in both ASCII and Vega-Lite (4 passing cases), including ranges 10,20,30 with only Low and numeric marker/line fallbacks. The saved update regression preserves a short label list; the frontend transformProps parity regression pins the same behavior. Focused backend: 450 passed; frontend: 9 suites / 122 passed; branch-wide pre-commit passed. Normal push completed; required exact-head CI is pending.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2014,6 +2099,462 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Typed contract for the ECharts Bullet visualization (viz_type ``bullet``).
+
+ Semantic field names are exposed to MCP clients while validation aliases and
+ the native adapter accept saved Explore ``form_data`` without weakening the
+ unknown-field checks that catch misspelled controls.
+ """
+
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description=(
+ "Numeric measure shown by each bullet bar. Use aggregate for a SIMPLE "
+ "metric, saved_metric=True for a dataset metric, or sql_expression "
+ "with a unique label."
+ ),
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Optional category hierarchy; the frontend renders one bullet row per "
+ "unique combination (native form_data: groupby). Omit to preserve a "
+ "saved hierarchy on update; pass [] to clear it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(
+ None,
+ description=(
+ "Structured WHERE filters. Native SIMPLE adhoc_filters are accepted; "
+ "free-form SQL filters are rejected."
+ ),
+ max_length=100,
+ )
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Optional Superset time range such as 'Last 30 days' or "
+ "'2025-01-01 : 2025-12-31'. Set temporal_column to choose its column."
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum grouped bullet rows returned by the query",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description=(
+ "Stable row ordering by a dimension name or by the metric's output "
+ "label/name. Native orderby pairs and order_by_cols JSON pairs are "
+ "accepted for saved-form-data round trips."
+ ),
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ data.setdefault("temporal_column", subject)
+ if isinstance(comparator, str) and comparator.casefold() != "no filter":
+ data.setdefault("time_range", comparator)
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
+ {"name": item} if isinstance(item, str) else item
+ for item in data[key]
+ ]
+ for key in ("order_by", "orderby", "order_by_cols"):
+ if key in data:
+ data[key] = cls._adapt_native_order_by(data[key])
+ cls._adapt_native_filters(data)
+ return data
+
+ @field_validator("ranges", "markers", "marker_lines", mode="before")
+ @classmethod
+ def tokenize_native_numeric_lists(cls, value: Any) -> Any:
+ """Parse numeric controls without creating values for empty tokens."""
+ if value is None:
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",") if token.strip()]
+ return value
+
+ @field_validator(
+ "range_labels", "marker_labels", "marker_line_labels", mode="before"
+ )
+ @classmethod
+ def tokenize_native_label_lists(cls, value: Any) -> Any:
+ """Parse label controls while preserving positional empty tokens."""
+ if value is None or value == "":
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",")]
+ return value
+
+ @field_validator("ranges", "markers", "marker_lines")
+ @classmethod
+ def reject_non_finite_values(cls, values: List[float]) -> List[float]:
+ if any(not math.isfinite(value) for value in values):
+ raise ValueError("Bullet thresholds and markers must be finite numbers")
+ return values
+
+ @field_validator("range_labels", "marker_labels", "marker_line_labels")
+ @classmethod
+ def validate_presentation_labels(cls, labels: List[str]) -> List[str]:
+ result: list[str] = []
+ for label in labels:
+ if "," in label:
+ raise ValueError(
+ "Bullet labels cannot contain commas because the frontend "
+ "comma-separated controls have no escaping"
+ )
+ if label == "":
+ result.append("")
+ continue
+ sanitized = sanitize_user_input(
+ label, "Bullet label", max_length=200, allow_empty=True
+ )
+ if sanitized is not None:
+ result.append(sanitized)
+ return result
+
+ @field_validator("time_range")
+ @classmethod
+ def sanitize_time_range(cls, value: str | None) -> str | None:
+ return sanitize_user_input(
+ value, "Time range", max_length=1000, allow_empty=True
+ )
+
+ @model_validator(mode="after")
+ def validate_roles_and_outputs(self) -> "BulletChartConfig": # noqa: C901
+ dimensions = self.dimensions or []
+ seen_names: set[str] = set()
+ for index, dimension in enumerate(dimensions):
+ _reject_sql_expression_on_dimension(dimension, f"dimensions[{index}]")
+ if dimension.saved_metric or dimension.aggregate:
+ raise ValueError(
+ f"dimensions[{index}] must be a physical dimension, not a metric"
+ )
+ name = dimension.name or ""
+ if name.casefold() in seen_names:
Review Comment:
Fixed in 1e3150c8ae8d8cb5ed1ace38de6159cf7ac2558c. Role/output collision checks use exact physical names, preserving Region and region (and a case-distinct metric alias). Dataset resolution still prefers exact names and rejects ambiguous non-exact lookup. Sort normalization resolves the original output role before canonicalization instead of storing case-folded dictionary keys that could overwrite one another.
Evidence: test_bullet_exact_case_dimensions_survive_normalization_and_query asserts both query columns, distinct sort targets, distinct result keys, and ambiguous lookup rejection. test_bullet_short_labels_and_case_distinct_dimensions_reach_fastmcp exercises both fields through saved/cached public previews in ASCII and Vega-Lite; the frontend transformProps regression verifies exact-key grouping. Exact duplicates still fail. Focused backend: 450 passed; frontend: 9 suites / 122 passed; branch-wide pre-commit passed. Normal push completed; required exact-head CI is pending.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4239672460
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1809,7 +1815,1060 @@ def generate_xy_vega_lite_preview(
spec["transform"] = [
{"fold": series_columns, "as": [series_field, value_field]}
]
- encoding["y"] = {"field": value_field, "type": "quantitative"}
+ # Vega-Lite stacks bar and area marks colored by a nominal field
+ # by default; follow the chart's saved ``stack`` control instead.
+ encoding["y"] = {
+ "field": value_field,
+ "type": "quantitative",
+ "stack": _xy_vega_lite_stack(form_data.get("stack")),
+ }
encoding["color"] = {"field": series_field, "type": "nominal"}
+ if (
+ spec.get("mark") == "bar"
+ and encoding["y"]["stack"] is None
+ and "x" in encoding
+ ):
+ # Unstacked bars sit side by side like Explore's grouped bars.
+ # Offset channels need a discrete band x scale, so temporal
+ # values stay date-formatted through a full-precision time unit.
+ x_encoding = encoding["x"]
+ if x_encoding.get("type") == "temporal":
+ x_encoding["timeUnit"] = "yearmonthdatehoursminutesseconds"
Review Comment:
Confirmed: `yearmonthdatehoursminutesseconds` drops milliseconds, so sub-second rows shared a band. Fixed in 44f32eafd26c892f303cf6b0b667e69c52eec7fe by switching to `yearmonthdatehoursminutessecondsmilliseconds`, which keeps each timestamp in its own band like Explore. Updated the comment and the pinning test, and added `test_wide_preview_unstacked_bars_keep_sub_second_timestamps_distinct` (fails on the old unit, passes now).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4228036517
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +997,443 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
+
+
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ offsets: list[Any] = []
+ if _time_comparison(form_data, query.get("metrics") or []):
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
- if (
- not isinstance(entry, (list, tuple))
- or len(entry) != 2
- or not isinstance(entry[0], str)
- or not entry[0]
- or not isinstance(entry[1], bool)
- ):
- raise ValueError(
- f"Gantt order_by_cols[{index}] must be [column, ascending_boolean]"
- )
- orderby.append([entry[0], entry[1]])
- columns: list[Any] = []
- seen: set[str] = set()
- for column in (
- start_time,
- end_time,
- category,
- *series_columns,
- *tooltip_columns,
- *(entry[0] for entry in orderby),
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ **(
+ {"isColumnReference": True}
+ if str(form_data.get("datasource", "")).endswith("__semantic_view")
+ else {}
+ ),
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
):
- key = utils_json.dumps(column, sort_keys=True, default=str)
- if key not in seen:
- seen.add(key)
- columns.append(column)
- return columns, list(raw_tooltip_metrics), orderby, series_columns
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
-def extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
- """Return the x_axis column name from form_data, or None if not set."""
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
x_axis = form_data.get("x_axis")
- if isinstance(x_axis, str) and x_axis:
- return x_axis
- if isinstance(x_axis, dict):
- col_name = x_axis.get("column_name")
- return col_name if isinstance(col_name, str) and col_name else None
- return None
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
+ **(
+ {"fill_time_range": True}
+ if form_data.get("resample_fill_time_range")
+ else {}
+ ),
+ },
+ }
+ )
+ rolling_type = form_data.get("rolling_type")
+ rolling_columns = (
+ [*offset_map.values(), *offset_map.keys()] if comparison else metric_labels
+ )
+ if rolling_type == "cumsum":
+ chain.append(
+ {
+ "operation": "cum",
+ "options": {
+ "operator": "sum",
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ elif rolling_type in {"sum", "mean", "std"}:
+ chain.append(
+ {
+ "operation": "rolling",
+ "options": {
+ "rolling_type": rolling_type,
+ "window": int(form_data.get("rolling_periods") or 1),
+ "min_periods": int(form_data.get("min_periods") or 0),
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ comparison_type = form_data.get("comparison_type")
+ if comparison and comparison_type != "values":
+ chain.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": list(offset_map.values()),
+ "compare_columns": list(offset_map.keys()),
+ "compare_type": comparison_type,
+ "drop_original_columns": True,
+ },
+ }
+ )
+ if complete_timeseries_contract and form_data.get("contributionMode"):
+ chain.append(
+ {
+ "operation": "contribution",
+ "options": {
+ "orientation": form_data["contributionMode"],
+ "time_shifts": offsets if comparison else [],
+ },
+ }
+ )
+ if comparison:
+ rename: dict[str, str | None] = {}
+ for shifted, metric in offset_map.items():
+ offset = next(
+ (item for item in offsets if shifted.endswith(f"__{item}")), None
+ )
+ source = (
+ shifted
+ if comparison_type == "values"
+ else f"{comparison_type}__{metric}__{shifted}"
+ )
+ rename[source] = f"{metric}, {offset}" if len(metrics) > 1 else offset
+ if rename:
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {"columns": rename, "level": 0, "inplace": True},
+ }
+ )
+ elif (
+ comparison_type not in {"difference", "percentage", "ratio"}
+ and x_label
+ and len(metrics) == 1
+ and (series_labels or len(offsets) > 1)
+ and form_data.get("truncate_metric") is not None
+ and form_data.get("truncate_metric")
+ ):
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {
+ "columns": {metric_labels[0]: None},
+ "level": 0,
+ "inplace": True,
+ },
+ }
+ )
+ if complete_timeseries_contract:
+ x_axis_sort = form_data.get("x_axis_sort")
+ x_axis_sort_asc = form_data.get("x_axis_sort_asc")
+ sortable_labels = [
+ label
+ for label in [
+ x_label,
+ *(
+ _metric_label(metric)
+ for metric in _as_list(form_data.get("metrics"))
+ ),
+ *(
+ _metric_label(metric)
+ for metric in _timeseries_extra_metrics(form_data)
+ ),
+ ]
+ if label
+ ]
+ if (
+ x_axis_sort is not None
+ and x_axis_sort_asc is not None
+ and x_axis_sort in sortable_labels
+ and not _as_list(form_data.get("groupby"))
+ ):
+ options: dict[str, Any] = {"ascending": x_axis_sort_asc}
+ if x_axis_sort == x_label:
+ options["is_sort_index"] = True
+ else:
+ options["by"] = x_axis_sort
+ chain.append({"operation": "sort", "options": options})
+ chain.append({"operation": "flatten"})
+
+ if complete_timeseries_contract and form_data.get("forecastEnabled") and x_label:
+ x_axis_grain = (
+ x_axis.get("timeGrain")
+ if isinstance(x_axis, dict)
+ and x_axis.get("expressionType") in {"SIMPLE", "SQL"}
+ else None
+ )
+ time_grain = (
+ x_axis_grain
+ or (query.get("extras") or {}).get("time_grain_sqla")
+ or form_data.get("time_grain_sqla")
+ or "P1D"
+ )
+ chain.append(
+ {
+ "operation": "prophet",
Review Comment:
Fixed. After merging master in 17db085b806ccd0a9ea55857672edfdafac34862, the compile check uses the shared builder in `compile.py`, which drops `forecastEnabled` (and `rolling_type`/`rolling_type_b`) from the bounded sample, so the Prophet step never runs on the two-row compile query; full data and preview queries keep it. In 91019555303cd0571a604e2b1688826fcfc84f74 I added `test_compile_sample_skips_forecast_for_grouped_series_on_one_timestamp`, which runs the real post-processing on East/West rows that share one day. With the forecast step kept in the sample it fails with "Forecast requires at least 2 data points"; with the fix it passes.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,151 @@ def fallback_vega_lite_preview(
return None
+# Extended ISO date/datetime text that the renderer's ``Date.parse`` accepts.
+# Python's ISO parser also accepts basic forms such as ``20250101`` and week
+# dates, which browsers reject, so those must not mark the axis temporal.
+_VEGA_TEMPORAL_TEXT = re.compile(
+ r"\d{4}-\d{2}-\d{2}"
+ r"(?:[T ]\d{2}:\d{2}(?::\d{2}(?:\.\d{1,9})?)?(?:Z|[+-]\d{2}:\d{2})?)?"
+)
+
+
+def _xy_pivot_x_type(values: list[Any]) -> str:
+ """Infer the Vega-Lite x type from every x value, not a character scan.
+
+ The renderer has no column metadata, so text is temporal only when each
+ value is an extended ISO date or datetime the renderer can parse; labels
+ such as ``New York`` and compact dates such as ``20250101`` stay nominal
+ instead of becoming unparseable dates.
+ """
+ present = [value for value in values if value is not None]
+ if not present:
+ return "nominal"
+ if all(
+ isinstance(value, (int, float)) and not isinstance(value, bool)
+ for value in present
+ ):
+ return "quantitative"
+ if all(
+ isinstance(value, (date, datetime))
+ or (
+ isinstance(value, str)
+ and _VEGA_TEMPORAL_TEXT.fullmatch(value) is not None
+ and _gantt_temporal_value(value) is not None
+ )
+ for value in present
+ ):
+ return "temporal"
+ return "nominal"
+
+
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list, _time_comparison
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ def vega_field(name: str) -> str:
+ """Escape a literal key so Vega-Lite does not read it as a nested path."""
+ return "".join("\\" + char if char in ".[]\\" else char for char in name)
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ series_labels = list(metric_labels)
+ if len(metric_labels) == 1 and _time_comparison(
+ form_data, _as_list(form_data.get("metrics"))
+ ):
+ # A single compared metric renames each shifted series to its bare
+ # offset, so those series carry the offset rather than the metric.
+ series_labels.extend(
+ str(offset) for offset in _as_list(form_data.get("time_compare"))
+ )
+ prefixes = {
+ spelling
+ for label in series_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(prefix + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(prefix + "__")
+ for prefix in prefixes
+ )
+ ]
+ if not fields and len(metric_labels) == 1 and form_data.get("truncate_metric"):
+ # A single truncated metric drops its label from the pivoted column
+ # names, so every non-x-axis column is one category series.
+ fields = [field for field in data[0] if field != x_axis]
+ if not fields:
+ return None
+ x_type = _xy_pivot_x_type([row.get(x_axis) for row in data])
+ return VegaLitePreview(
+ specification={
+ "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
+ "data": {"values": data},
+ "transform": [
+ {
+ "fold": [vega_field(field) for field in fields],
+ "as": ["__mcp_xy_series", "__mcp_xy_value"],
+ }
+ ],
+ "mark": mark,
+ "encoding": {
+ "x": {"field": vega_field(x_axis), "type": x_type, "title": x_axis},
+ "y": {
Review Comment:
Fixed in 91019555303cd0571a604e2b1688826fcfc84f74. After the merge the fold lives in `generate_xy_vega_lite_preview`, and it had the same gap. The y encoding now follows the saved `stack` control: `stack: null` when unstacked, `zero` for Stack, and `normalize` for Expand. `test_wide_preview_fold_follows_the_saved_stack_control` covers None/False/Stack/True/Expand on a grouped bar with East=10 and West=20, and fails without the change because the encoding has no `stack` key.
##########
superset/mcp_service/semantic_layer/tool/get_table.py:
##########
@@ -366,23 +372,24 @@ def _build_response(
is_builtin: bool,
display_name: str,
query_result: dict[str, Any],
+ data: list[dict[str, Any]],
+ raw_columns: list[str],
+ coltypes: list[int | GenericDataType],
query_duration_ms: int,
warnings: list[str],
temporal_columns: set[str] | None = None,
valid_grains: dict[str, dict[str, str]] | None = None,
) -> GetTableResponse:
"""Format the query result into a GetTableResponse."""
- data = query_result.get("data", [])
- raw_columns = query_result.get("colnames", [])
cache_status = get_cache_status_from_result(
query_result, force_refresh=request.force_refresh
)
-
if not data:
+ columns_meta = format_data_columns(data, raw_columns, coltypes)
Review Comment:
Fixed by the merge of master (17db085b806ccd0a9ea55857672edfdafac34862): `_build_response` now computes `result_temporal_columns` once and passes it to `format_data_columns` before the empty-result branch, so both paths type the grain columns the same way. In 91019555303cd0571a604e2b1688826fcfc84f74 I added `test_get_table_empty_result_keeps_temporal_grain_column_type` (dimensions `[metric_time]`, `time_grain=P1M`, no rows). It asserts `["temporal", "string"]`, and it fails with `["string", "string"]` if the overrides are skipped for empty data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4235304737
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1809,7 +1815,1041 @@ def generate_xy_vega_lite_preview(
spec["transform"] = [
{"fold": series_columns, "as": [series_field, value_field]}
]
- encoding["y"] = {"field": value_field, "type": "quantitative"}
+ # Vega-Lite stacks bar and area marks colored by a nominal field
+ # by default; follow the chart's saved ``stack`` control instead.
+ encoding["y"] = {
+ "field": value_field,
+ "type": "quantitative",
+ "stack": _xy_vega_lite_stack(form_data.get("stack")),
+ }
encoding["color"] = {"field": series_field, "type": "nominal"}
return preview
+
+
+def _xy_vega_lite_stack(stack: Any) -> str | None:
+ """Map the ECharts timeseries ``stack`` control to a Vega-Lite stack."""
+ if not stack:
+ return None
+ return "normalize" if stack == "Expand" else "zero"
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+_MAX_BULLET_FIELDS = 256
+
+
+_MAX_BULLET_FIELD_BYTES = 1000
+
+
+_MAX_BULLET_TEXT_BYTES = 2000
+
+
+# ECMAScript WhiteSpace and LineTerminator characters used by trim/Number.
+_JAVASCRIPT_WHITESPACE = (
+ "\t\n\v\f\r \u00a0\u1680\u2000\u2001\u2002\u2003\u2004"
+ "\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000\ufeff"
+)
+
+
+_MAX_BULLET_TOKENS = 256
+
+
+_ENUM_SCALAR_TYPES = (str, int, float, bool, Decimal)
+
+
+class BulletOutputError(ValueError):
+ """A Bullet query result cannot be rendered without guessing its roles."""
+
+ def __init__(self, message: str, error_type: str = "MalformedBulletOutput") -> None:
+ super().__init__(message)
+ self.error_type = error_type
+
+
+@dataclass(frozen=True)
+class BulletRenderModel:
+ """Strict, frontend-aligned data and presentation roles for one preview."""
+
+ rows: list[dict[str, Any]]
+ metric_field: str
+ dimensions: list[str]
+ measures: list[float]
+ ranges: list[float]
+ range_labels: list[str]
+ markers: list[float]
+ marker_labels: list[str]
+ marker_lines: list[float]
+ marker_line_labels: list[str]
+ y_axis_format: str
+ show_labels: bool
+ show_legend: bool
+
+
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ # getColumnLabel: an adhoc column without a label is keyed by its SQL.
+ for key in ("label", "sqlExpression", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if type(value) in {list, tuple, dict}:
+ data, failure = first_query_data(
+ {"queries": [{"data": [{"value": value}]}]}, temporal_json_numbers=True
+ )
+ if failure is not None or data is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} "
+ "has an invalid or unbounded container value"
+ )
+ normalized = data[0]["value"]
+ reason = None
+ elif _is_chart_data_temporal_scalar(value := _safe_enum_backing(value)):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_scalar(value)
+ if (
+ reason is None
+ and normalized is None
+ and value is not None
+ and type(value) in {float, np.float16, np.float32, np.float64}
+ and math.isinf(float(value))
+ ):
+ # NaN keeps the shared null normalization, but an infinite source
+ # value has no faithful category text and is rejected.
+ reason = "contains a non-finite number"
+ if (
+ reason is None
+ and type(normalized) is str
+ and _bounded_utf8_length(normalized, _MAX_BULLET_TEXT_BYTES) is None
+ ):
+ reason = "exceeds the size limit"
+ if reason is not None:
+ if reason == "an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif not reason.startswith(("contains ", "exceeds ", "has ")):
+ reason = f"contains {reason}"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if value_type in {list, dict}:
+ text = _bullet_container_category_text(normalized, dimension, row_index)
+ elif normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_container_category_text(value: Any, dimension: str, row_index: int) -> str:
+ """Stringify validated containers like JavaScript with a bounded text budget."""
+ from superset.mcp_service.chart.query_result import _bounded_utf8_length
+
+ if type(value) is dict:
+ return "[object Object]"
+ parts: list[str] = []
+ size = 0
+ for index, item in enumerate(value):
+ if type(item) in {list, dict}:
+ text = _bullet_container_category_text(item, dimension, row_index)
+ else:
+ text = (
+ ""
+ if item is None
+ else _bullet_category_value(item, dimension, row_index)[1]
+ )
+ text_size = _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES)
+ size += (text_size if text_size is not None else _MAX_BULLET_TEXT_BYTES + 1) + (
+ index > 0
+ )
+ if size > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ parts.append(text)
+ return ",".join(parts)
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
Review Comment:
Good catch, confirmed: 100 labels x 200 chars (and also 100 numeric thresholds serialized as e.g. `-2.2250738585072014e-308`, ~2.5k chars) passed schema validation but tripped the 2000-char whole-control check. Fixed in 29b4469f40f418a1d8d20d2307eb58e8f53e6930: the joined-control bound is now `_MAX_BULLET_CONTROL_CHARS = _MAX_BULLET_TOKENS * (_MAX_BULLET_TEXT_BYTES + 1)`, i.e. the item cap times the per-token cap plus separators, applied to both the label tokenizer and the numeric control tokenizer; per-token and item-count limits are unchanged. Regression: `test_bullet_schema_maximum_presentation_controls_render` maps schema-maximum configs for all six controls and renders them.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -3939,6 +4058,477 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ # Saved query roles are internal sort context, never typed authoring input.
+ # Keeping them out of dimensions preserves omission and native SQL shapes.
+ _inherited_groupby: list[str | dict[str, Any]] | None = PrivateAttr(default=None)
+
+ @property
+ def order_dimensions(self) -> Sequence[ColumnRef | str | dict[str, Any]]:
+ """Return authored dimensions or the inherited native sort hierarchy."""
+ if self.dimensions is None and self._inherited_groupby is not None:
+ return self._inherited_groupby
+ return self.dimensions or []
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
Review Comment:
Confirmed and fixed in 29b4469f40f418a1d8d20d2307eb58e8f53e6930. `adapt_native_form_data` now adapts every present ordering alias (`order_by`, `orderby`, `order_by_cols`) and concatenates them in key order into a single `order_by` list before field selection, matching `_normalize_bullet_query_aliases`, so AliasChoices never shadows one with another. Regression: `test_bullet_native_order_aliases_are_concatenated_not_shadowed` covers your exact example (`orderby: []` + `order_by_cols: ['["sales", false]']` keeps the descending-sales sort) and the both-populated concatenation order.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -3939,6 +4058,477 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ # Saved query roles are internal sort context, never typed authoring input.
+ # Keeping them out of dimensions preserves omission and native SQL shapes.
+ _inherited_groupby: list[str | dict[str, Any]] | None = PrivateAttr(default=None)
+
+ @property
+ def order_dimensions(self) -> Sequence[ColumnRef | str | dict[str, Any]]:
+ """Return authored dimensions or the inherited native sort hierarchy."""
+ if self.dimensions is None and self._inherited_groupby is not None:
+ return self._inherited_groupby
+ return self.dimensions or []
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ temporal_pairs: list[tuple[str, str]] = []
+ inert_subjects: set[str] = set()
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ if not isinstance(comparator, str) or not comparator:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs a range"
+ )
+ if comparator.casefold() == "no filter":
+ inert_subjects.add(subject)
+ else:
+ temporal_pairs.append((subject, comparator))
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ if len(temporal_pairs) > 1:
+ raise ValueError(
+ "Multiple active native temporal filters cannot be represented by "
+ "a single temporal_column/time_range pair"
+ )
+ if temporal_pairs:
+ subject, comparator = temporal_pairs[0]
+ if data.get("temporal_column") not in (None, subject) or data.get(
+ "time_range"
+ ) not in (None, "No filter", comparator):
+ raise ValueError(
+ "Native temporal filter conflicts with temporal_column/time_range"
+ )
+ data["temporal_column"] = subject
+ data["time_range"] = comparator
+ elif len(inert_subjects) == 1:
+ data.setdefault("temporal_column", next(iter(inert_subjects)))
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ # Null means no hierarchy was supplied, not a conflicting alias or a
+ # request to clear saved dimensions. Only an explicit [] clears them.
+ for key in ("dimensions", "groupby"):
+ if data.get(key) is None:
+ data.pop(key, None)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
Review Comment:
Confirmed and fixed in 29b4469f40f418a1d8d20d2307eb58e8f53e6930. `_normalize_chart_request_input` now sets the shared `_NATIVE_FORM_DATA_MARKER` for `viz_type: "bullet"` (as it does for `sunburst_v2`), and `BulletChartConfig.adapt_native_form_data` pops the marker and treats it like `viz_type == "bullet"`, so the lenient native numeric parsing runs on both entry points. Typed `chart_type: "bullet"` requests keep strict float validation. Regression: `test_bullet_native_numeric_controls_parse_alike_on_request_paths` checks GenerateChartRequest, UpdateChartRequest and UpdateChartPreviewRequest yield the same markers/ranges as direct BulletChartConfig validation (`markers: "0x64"` -> [100.0]) and that the typed path still rejects it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3975187132
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1007,3 +1007,19 @@ The callable must be cheap and in-process (consult already-loaded feature flags
- **[MCP Integration](/developer-docs/extensions/mcp)** -- Build custom MCP tools and prompts via Superset extensions
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
Review Comment:
Added `bullet` to the supported chart-plugin list so it matches the registered plugin and the compatibility section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3918794721
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1065,6 +1121,660 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
+ raise ValueError(
+ "MCP temporal binding provenance must match exactly one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found {len(matches)}"
+ )
+ return matches[0]
+
+
+def _append_or_replace_filter(filters: list[Any], filter_: Any) -> None:
+ """Append a filter, replacing the same native role when identifiable."""
+ identity = _filter_identity(filter_)
+ if identity is None:
+ if filter_ not in filters:
+ filters.append(filter_)
+ return
+ filters[:] = [item for item in filters if _filter_identity(item) != identity]
+ filters.append(filter_)
+
+
+_NATIVE_TEMPORAL_ROLE_FIELDS: dict[str, frozenset[str]] = {
+ # Typed ``x`` is persisted as native x_axis/granularity_sqla for XY and
+ # Mixed Timeseries. Waterfall exposes the typed field as ``x_axis``.
+ "x_axis": frozenset({"x", "x_axis"}),
+ "granularity_sqla": frozenset({"x", "x_axis", "temporal_column"}),
+ # Chart plugins may designate a chart-specific query role as the implicit
+ # dashboard-time subject.
+ "start_time": frozenset({"start_time"}),
+}
+
+
+def _native_temporal_subject_changed(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Mapping[str, Any],
+ explicit_fields: set[str],
+) -> bool:
+ """Return whether an authoritative native temporal role was replaced.
+
+ Mapping a partial update can propose a dataset fallback binding even when
+ the caller only changed filters. That proposal is not authoritative. A
+ changed x/granularity/chart-specific role is authoritative only when its
+ corresponding typed field was actually supplied.
+ """
+ for native_key, typed_fields in _NATIVE_TEMPORAL_ROLE_FIELDS.items():
+ if explicit_fields.isdisjoint(typed_fields):
+ continue
+ existing_value = existing_form_data.get(native_key)
+ incoming_value = new_form_data.get(native_key)
+ if existing_value != incoming_value:
+ return True
+ return False
+
+
+def _native_temporal_binding(
+ form_data: Mapping[str, Any], filters: list[Any]
+) -> tuple[str | None, dict[str, Any] | None]:
+ """Resolve one binding for a trusted native temporal role, if present."""
+ for native_key in _NATIVE_TEMPORAL_ROLE_FIELDS:
+ subject = form_data.get(native_key)
+ if not isinstance(subject, str) or not subject:
+ continue
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) > 1:
+ raise ValueError(
+ "An authoritative native temporal subject must match at most one "
+ f"TEMPORAL_RANGE filter for subject {subject!r}; found "
+ f"{len(matches)}"
+ )
+ if matches:
+ return subject, matches[0]
+ return None, None
+
+
+def merge_update_form_data( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply the shared omission/provenance contract for chart updates.
+
+ Mapper-generated neutral temporal bindings are infrastructure, not evidence
+ that the caller supplied ``filters`` or changed a saved time-range binding.
+ This helper is used by immediate saves, preview-first saved updates, and
+ cached-preview updates so omission, clear, replacement, and temporal
+ overrides have identical behavior.
+ """
+ existing_filters = list(existing_form_data.get("adhoc_filters") or [])
+ incoming_filters = list(new_form_data.get("adhoc_filters") or [])
+ existing_subject = existing_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ incoming_subject = new_form_data.get(MCP_DASHBOARD_TIME_FILTER_SUBJECT)
+ existing_binding = _temporal_binding_filter(existing_filters, existing_subject)
+ incoming_binding = _temporal_binding_filter(incoming_filters, incoming_subject)
+
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ filters_explicit = "filters" in explicit_fields
+ range_explicit = "time_range" in explicit_fields
+ subject_explicit = "temporal_column" in explicit_fields
+ native_subject_changed = _native_temporal_subject_changed(
+ existing_form_data, new_form_data, explicit_fields
+ )
+ subject_authoritative = subject_explicit or native_subject_changed
+ if incoming_binding is None:
+ native_subject, native_binding = _native_temporal_binding(
+ new_form_data, incoming_filters
+ )
+ if native_binding is not None:
+ incoming_subject = native_subject
+ incoming_binding = native_binding
+ incoming_user_filters = [
+ filter_ for filter_ in incoming_filters if filter_ is not incoming_binding
+ ]
+ temporal_explicit = range_explicit or subject_authoritative
+
+ chosen_binding: dict[str, Any] | None = None
+ chosen_subject: Any = None
+ if not filters_explicit:
+ # Omission is byte-faithful: keep the native sequence in its exact order,
+ # including SQL/HAVING objects and a provenance-owned binding at any index.
+ merged_filters = list(existing_filters)
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif existing_binding is not None:
+ # A range-only update belongs to the saved subject, even when
+ # mapping the partial config proposed the dataset main_dttm.
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif existing_binding is not None:
+ # Subject-only replacement preserves the saved active or
+ # neutral range instead of resetting it to No filter.
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if existing_binding is not None:
+ binding_index = next(
+ index
+ for index, filter_ in enumerate(merged_filters)
+ if filter_ is existing_binding
+ )
+ if chosen_binding is None:
+ merged_filters.pop(binding_index)
+ else:
+ # A temporal override changes infrastructure in place instead
+ # of moving it past surrounding native filters.
+ merged_filters[binding_index] = chosen_binding
+ elif chosen_binding is not None:
+ merged_filters.append(chosen_binding)
+ else:
+ # An explicit filter array replaces the saved native sequence. The mapper
+ # deliberately emits [] for an explicit clear; otherwise retain its
+ # generated temporal binding after the replacement filters.
+ merged_filters = list(incoming_user_filters)
+ if incoming_user_filters or temporal_explicit:
+ if subject_authoritative:
+ chosen_binding = incoming_binding
+ chosen_subject = incoming_subject
+ elif range_explicit and existing_binding is not None:
+ chosen_binding = dict(existing_binding)
+ chosen_subject = existing_subject
+ else:
+ # A filter-only replacement keeps the saved provenance binding.
+ # The mapper's incoming binding may merely be a dataset fallback
+ # and must not reset the saved subject or active range.
+ chosen_binding = existing_binding
+ chosen_subject = existing_subject
+ if chosen_binding is not None:
+ chosen_binding = dict(chosen_binding)
+ if range_explicit:
+ chosen_binding["comparator"] = (
+ getattr(config, "time_range", None) or NO_TIME_RANGE
+ )
+ elif subject_authoritative and existing_binding is not None:
+ chosen_binding["comparator"] = existing_binding.get(
+ "comparator", NO_TIME_RANGE
+ )
+ if chosen_binding is not None:
+ _append_or_replace_filter(merged_filters, chosen_binding)
+
+ # Materialize exactly when saved state had the key or the caller made the
+ # controls authoritative. An omitted update must not turn a missing native
+ # filter key into [] merely because its mapper proposed a neutral binding.
+ if filters_explicit or "adhoc_filters" in existing_form_data or temporal_explicit:
+ new_form_data["adhoc_filters"] = merged_filters
+ else:
+ new_form_data.pop("adhoc_filters", None)
+ if chosen_binding is not None and isinstance(chosen_subject, str):
+ new_form_data[MCP_DASHBOARD_TIME_FILTER_SUBJECT] = chosen_subject
+ else:
+ new_form_data.pop(MCP_DASHBOARD_TIME_FILTER_SUBJECT, None)
+
+ merge_bullet_form_data(existing_form_data, new_form_data)
+
+
+def _currency_form_value(value: CurrencyFormat | None) -> dict[str, str] | None:
+ """Return the native value for an explicitly supplied currency control."""
+ return value.to_form_data() if value is not None else None
+
+
+def _column_names(value: Sequence[ColumnRef] | None) -> list[str | None] | None:
+ """Return a native column-name list while retaining an explicit null."""
+ return [column.name for column in value] if value is not None else None
+
+
+def _table_sort_value(value: Sequence[str | SortByConfig] | None) -> list[str] | None:
+ """Return the native Table sort control for an explicit typed value."""
+ if value is None:
+ return None
+ return [
+ json.dumps(
+ [entry.column, entry.ascending]
+ if isinstance(entry, SortByConfig)
+ else [entry, False]
+ )
+ for entry in value
+ ]
+
+
+def _table_column_config_value(value: Any) -> dict[str, Any] | None:
+ """Return Table column config without losing an explicit null or empty map."""
+ if value is None:
+ return None
+ return {
+ label: column.model_dump(by_alias=True, exclude_unset=True)
+ for label, column in value.items()
+ }
+
+
+# Mappers intentionally omit optional controls so fresh charts use the frontend
+# defaults. During a same-viz update, however, an explicitly supplied false,
+# null, or empty value must block preservation of the saved native key. Keep the
+# typed-to-native relationship declarative so every update path shares it.
+_FormValueConverter = Callable[[Any], Any]
+_FormControlMap = dict[str, tuple[str, _FormValueConverter]]
+
+_COMMON_EXPLICIT_FORM_CONTROLS: _FormControlMap = {
+ "color_scheme": ("color_scheme", lambda value: value),
+ "currency_format": ("currency_format", _currency_form_value),
+ "show_value": ("show_value", lambda value: value),
+}
+
+_CHART_EXPLICIT_FORM_CONTROLS: dict[str, _FormControlMap] = {
+ "table": {
+ "sort_by": ("order_by_cols", _table_sort_value),
+ "column_config": ("column_config", _table_column_config_value),
+ },
+ "xy": {
+ "group_by": ("groupby", _column_names),
+ "series_limit": ("series_limit", lambda value: value),
+ "stacked": ("stack", lambda value: "Stack" if value else None),
+ "orientation": ("orientation", lambda value: value),
+ "legend_orientation": ("legendOrientation", lambda value: value),
+ "x_axis_time_format": ("x_axis_time_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "mixed_timeseries": {
+ "group_by": ("groupby", _column_names),
+ "group_by_secondary": ("groupby_b", _column_names),
+ "currency_format_secondary": (
+ "currency_format_secondary",
+ _currency_form_value,
+ ),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "waterfall": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ },
+ "big_number": {
+ "subheader": ("subheader", lambda value: value),
+ "y_axis_format": ("y_axis_format", lambda value: value),
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "compare_lag": ("compare_lag", lambda value: value),
+ "time_format": ("time_format", lambda value: value),
+ "aggregation": ("aggregation", lambda value: value),
+ },
+ "handlebars": {
+ "style_template": ("styleTemplate", lambda value: value),
+ "columns": ("all_columns", _column_names),
+ "groupby": ("groupby", _column_names),
+ "metrics": ("metrics", _column_names),
+ },
+ "pivot_table": {
+ "date_format": ("date_format", lambda value: value),
+ },
+ "interactive_pivot": {
+ "time_grain": ("time_grain_sqla", lambda value: value),
+ "series_limit": ("series_limit", lambda value: value),
+ "date_format": ("date_format", lambda value: value),
+ "column_sort": ("colOrder", lambda value: value),
+ },
+}
+
+
+def _apply_explicit_form_controls( # noqa: C901
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Apply typed controls whose mapper omission represents a native clear."""
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ controls = {
+ **_COMMON_EXPLICIT_FORM_CONTROLS,
+ **_CHART_EXPLICIT_FORM_CONTROLS.get(config.chart_type, {}),
+ }
+ for field_name, (native_key, convert) in controls.items():
+ if field_name in explicit_fields:
+ converted = convert(getattr(config, field_name))
+ is_clear = (
+ converted is None
+ or converted is False
+ or converted == ""
+ or converted in ([], {})
+ )
+ if is_clear:
+ new_form_data[native_key] = converted
+
+ axis_controls = {
+ "xy": (
+ ("x_axis", "x_axis_title", "x_axis_format", None),
+ ("y_axis", "y_axis_title", "y_axis_format", "y_axis_scale"),
+ ),
+ "mixed_timeseries": (
+ ("x_axis", "xAxisTitle", "x_axis_time_format", None),
+ ("y_axis", "yAxisTitle", "y_axis_format", "logAxis"),
+ (
+ "y_axis_secondary",
+ "yAxisTitleSecondary",
+ "y_axis_format_secondary",
+ "logAxisSecondary",
+ ),
+ ),
+ }
+ if config.chart_type in axis_controls:
+ for field_name, title_key, format_key, scale_key in axis_controls[
+ config.chart_type
+ ]:
+ if field_name not in explicit_fields:
+ continue
+ axis = getattr(config, field_name)
+ if axis is None:
+ new_form_data[title_key] = None
+ new_form_data[format_key] = None
+ if scale_key:
+ new_form_data[scale_key] = None
+ continue
+ axis_fields = set(axis.model_fields_set)
+ if "title" in axis_fields:
+ new_form_data[title_key] = axis.title
+ if "format" in axis_fields:
+ new_form_data[format_key] = axis.format
+ if scale_key and "scale" in axis_fields:
+ new_form_data[scale_key] = axis.scale
+
+ if config.chart_type == "xy" and "legend" in explicit_fields:
+ legend = config.legend
+ if legend is None:
+ new_form_data["show_legend"] = None
+ new_form_data["legendOrientation"] = None
+ else:
+ legend_fields = set(legend.model_fields_set)
+ if "show" in legend_fields:
+ new_form_data["show_legend"] = legend.show
+ if "position" in legend_fields:
+ new_form_data["legendOrientation"] = legend.position
+
+ if config.chart_type == "interactive_pivot":
+ if "temporal_column" in explicit_fields and config.temporal_column is None:
+ new_form_data["granularity_sqla"] = None
+ new_form_data["temporal_columns_lookup"] = None
+ if (
+ "series_limit_metric" in explicit_fields
+ and config.series_limit_metric is None
+ ):
+ new_form_data["series_limit_metric"] = None
+ if "comparison_period" in explicit_fields and config.comparison_period is None:
+ new_form_data["time_compare"] = None
+ if "comparison_type" in explicit_fields and config.comparison_type is None:
+ new_form_data["comparison_type"] = None
+
+ # A Waterfall axis replacement cannot inherit a bucket belonging to the old
+ # temporal subject. Grain omission preserves only while the axis is stable;
+ # explicit null is already handled by the declarative control map above.
+ if (
+ config.chart_type == "waterfall"
+ and existing_form_data.get("x_axis") != new_form_data.get("x_axis")
+ and "time_grain" not in explicit_fields
+ ):
+ new_form_data["time_grain_sqla"] = None
+
+
+def merge_same_viz_form_data(
+ existing_form_data: Mapping[str, Any],
+ new_form_data: Dict[str, Any],
+ config: ChartConfig,
+) -> None:
+ """Preserve saved controls that the typed mapper does not represent.
+
+ The typed MCP surface deliberately models a bounded subset of every Explore
+ control panel. For a replacement within the exact same native ``viz_type``,
+ keys absent from the mapper therefore represent omitted controls and retain
+ their saved values. Mapper output and the chart-specific merge helpers run
+ first and remain authoritative, including explicit empty, false, null, and
+ nested values.
+
+ No generic state crosses a visualization boundary. This prevents query-role
+ keys from the previous plugin (for example ``metric`` or ``groupby``) from
+ leaking into a different plugin whose role contract is unrelated.
+ """
+ existing_viz_type = existing_form_data.get("viz_type")
+ if not isinstance(existing_viz_type, str) or existing_viz_type != new_form_data.get(
+ "viz_type"
+ ):
+ return
+
+ _apply_explicit_form_controls(existing_form_data, new_form_data, config)
+
+ for key, value in existing_form_data.items():
+ if key == MCP_DASHBOARD_TIME_FILTER_SUBJECT:
+ # merge_update_form_data owns this provenance marker. Its absence
+ # may be an intentional subject clear and must not be undone by the
+ # generic preservation layer.
+ continue
+ if key not in new_form_data:
+ new_form_data[key] = value
+
+
+def validate_merged_bullet_form_data(
+ form_data: Mapping[str, Any],
+ update_config: ChartConfig | None = None,
+) -> BulletChartConfig | None:
+ """Validate final Bullet state without reclassifying preserved filters.
+
+ The typed Bullet surface intentionally creates only SIMPLE WHERE filters,
+ while saved Explore state may legitimately contain SQL WHERE or SIMPLE
+ HAVING filters. When an update omitted ``filters``, those native objects
+ came from the saved state and are validated by the form-data/query layer;
+ removing them only from this schema-validation copy avoids pretending they
+ were newly supplied typed filters. Explicit filter replacements, including
+ ``[]``, still take the strict native-to-typed path.
+ """
+ if form_data.get("viz_type") != "bullet":
+ return None
+ validation_data = dict(form_data)
+ if isinstance(update_config, BulletChartConfig) and (
Review Comment:
A dataset-only Bullet rebind has no `BulletChartConfig`, so this preservation exception does not run and existing SQL/HAVING filters are adapted as newly supplied typed filters. That makes `update_chart({dataset_id: ...})` reject otherwise valid saved Bullet charts before the rebind compiler can validate them. Should the config-less rebind path preserve those native filters too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r3936395644
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2014,6 +2099,462 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Typed contract for the ECharts Bullet visualization (viz_type ``bullet``).
+
+ Semantic field names are exposed to MCP clients while validation aliases and
+ the native adapter accept saved Explore ``form_data`` without weakening the
+ unknown-field checks that catch misspelled controls.
+ """
+
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description=(
+ "Numeric measure shown by each bullet bar. Use aggregate for a SIMPLE "
+ "metric, saved_metric=True for a dataset metric, or sql_expression "
+ "with a unique label."
+ ),
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Optional category hierarchy; the frontend renders one bullet row per "
+ "unique combination (native form_data: groupby). Omit to preserve a "
+ "saved hierarchy on update; pass [] to clear it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(
+ None,
+ description=(
+ "Structured WHERE filters. Native SIMPLE adhoc_filters are accepted; "
+ "free-form SQL filters are rejected."
+ ),
+ max_length=100,
+ )
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Optional Superset time range such as 'Last 30 days' or "
+ "'2025-01-01 : 2025-12-31'. Set temporal_column to choose its column."
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum grouped bullet rows returned by the query",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description=(
+ "Stable row ordering by a dimension name or by the metric's output "
+ "label/name. Native orderby pairs and order_by_cols JSON pairs are "
+ "accepted for saved-form-data round trips."
+ ),
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ data.setdefault("temporal_column", subject)
+ if isinstance(comparator, str) and comparator.casefold() != "no filter":
+ data.setdefault("time_range", comparator)
+ continue
+ if not isinstance(operator, str):
+ raise ValueError(f"adhoc_filters[{index}] needs an operator")
+ operator_map = {
+ "==": "=",
+ "EQUALS": "=",
+ "NOT_EQUALS": "!=",
+ "LESS_THAN": "<",
+ "LESS_THAN_OR_EQUAL": "<=",
+ "GREATER_THAN": ">",
+ "GREATER_THAN_OR_EQUAL": ">=",
+ "NOT_IN": "NOT IN",
+ "IS_NULL": "IS NULL",
+ "IS_NOT_NULL": "IS NOT NULL",
+ }
+ operator = operator_map.get(operator, operator)
+ filters.append({"column": subject, "op": operator, "value": comparator})
+ data["filters"] = filters
+
+ @model_validator(mode="before")
+ @classmethod
+ def adapt_native_form_data(cls, raw: Any) -> Any: # noqa: C901
+ """Accept recognized saved Bullet form_data and reject ambiguous state."""
+ if not isinstance(raw, dict):
+ return raw
+ data = dict(raw)
+ if "dimensions" in data and "groupby" in data:
+ dimensions = cls._canonical_dimension_alias(
+ data["dimensions"], "dimensions"
+ )
+ groupby = cls._canonical_dimension_alias(data["groupby"], "groupby")
+ if dimensions != groupby:
+ raise ValueError(
+ "Conflicting Bullet dimension aliases: 'dimensions' and "
+ "native 'groupby' must identify the same physical columns in "
+ "the same order; provide only one or make them equivalent"
+ )
+ # Avoid relying on AliasChoices precedence or JSON key order.
+ data.pop("groupby")
+ if data.get("viz_type") == "bullet":
+ data.setdefault("chart_type", "bullet")
+ data.pop("viz_type", None)
+ for key in (
+ "annotation_layers",
+ "dashboards",
+ "datasource",
+ "datasource_id",
+ "datasource_type",
+ "extra_form_data",
+ "slice_id",
+ "slice_name",
+ ):
+ data.pop(key, None)
+
+ if (marker_key := "_mcp_dashboard_time_filter_subject") in data:
+ marker = data.pop(marker_key)
+ if not isinstance(marker, str) or not marker:
+ raise ValueError(f"{marker_key} must be a physical column name")
+ raw_filters = data.get("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError(
+ f"{marker_key} requires an adhoc_filters array containing its "
+ "generated binding"
+ )
+ provenance_matches = [
+ filter_
+ for filter_ in raw_filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == marker
+ and filter_.get("operator") == "TEMPORAL_RANGE"
+ ]
+ if len(provenance_matches) != 1:
+ raise ValueError(
+ f"{marker_key} must match exactly one TEMPORAL_RANGE filter "
+ f"for subject {marker!r}; found {len(provenance_matches)}"
+ )
+ data.setdefault("temporal_column", marker)
+
+ if "metric" in data:
+ data["metric"] = cls._adapt_native_metric(data["metric"])
+ for key in ("groupby", "dimensions"):
+ if key in data:
+ if not isinstance(data[key], list):
+ raise ValueError(f"{key} must be an array")
+ data[key] = [
+ {"name": item} if isinstance(item, str) else item
+ for item in data[key]
+ ]
+ for key in ("order_by", "orderby", "order_by_cols"):
+ if key in data:
+ data[key] = cls._adapt_native_order_by(data[key])
+ cls._adapt_native_filters(data)
+ return data
+
+ @field_validator("ranges", "markers", "marker_lines", mode="before")
+ @classmethod
+ def tokenize_native_numeric_lists(cls, value: Any) -> Any:
+ """Parse numeric controls without creating values for empty tokens."""
+ if value is None:
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",") if token.strip()]
+ return value
+
+ @field_validator(
+ "range_labels", "marker_labels", "marker_line_labels", mode="before"
+ )
+ @classmethod
+ def tokenize_native_label_lists(cls, value: Any) -> Any:
+ """Parse label controls while preserving positional empty tokens."""
+ if value is None or value == "":
+ return []
+ if isinstance(value, str):
+ return [token.strip() for token in value.split(",")]
+ return value
+
+ @field_validator("ranges", "markers", "marker_lines")
+ @classmethod
+ def reject_non_finite_values(cls, values: List[float]) -> List[float]:
+ if any(not math.isfinite(value) for value in values):
+ raise ValueError("Bullet thresholds and markers must be finite numbers")
+ return values
+
+ @field_validator("range_labels", "marker_labels", "marker_line_labels")
+ @classmethod
+ def validate_presentation_labels(cls, labels: List[str]) -> List[str]:
+ result: list[str] = []
+ for label in labels:
+ if "," in label:
+ raise ValueError(
+ "Bullet labels cannot contain commas because the frontend "
+ "comma-separated controls have no escaping"
+ )
+ if label == "":
+ result.append("")
+ continue
+ sanitized = sanitize_user_input(
+ label, "Bullet label", max_length=200, allow_empty=True
+ )
+ if sanitized is not None:
+ result.append(sanitized)
+ return result
+
+ @field_validator("time_range")
+ @classmethod
+ def sanitize_time_range(cls, value: str | None) -> str | None:
+ return sanitize_user_input(
+ value, "Time range", max_length=1000, allow_empty=True
+ )
+
+ @model_validator(mode="after")
+ def validate_roles_and_outputs(self) -> "BulletChartConfig": # noqa: C901
+ dimensions = self.dimensions or []
+ seen_names: set[str] = set()
+ for index, dimension in enumerate(dimensions):
+ _reject_sql_expression_on_dimension(dimension, f"dimensions[{index}]")
+ if dimension.saved_metric or dimension.aggregate:
+ raise ValueError(
+ f"dimensions[{index}] must be a physical dimension, not a metric"
+ )
+ name = dimension.name or ""
+ if name.casefold() in seen_names:
Review Comment:
This treats `Region` and `region` as duplicate Bullet dimensions before dataset-aware resolution can preserve their exact physical names. Databases with quoted case-distinct columns can return both fields, and the frontend accesses each exact result key, so those charts are rejected even though they can be queried and rendered. Could role validation keep exact names distinct and reserve case-folding for ambiguous lookup only?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5508855318
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5497161238
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4178843216
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,439 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ # Match QueryObject's guarded legacy saved-metric normalization.
+ if not ({"sqlExpression", "aggregate", "column"} & value.keys()):
+ label = value.get("label")
+ if isinstance(label, str) and label:
+ return "saved_metric", label
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
+ viz_type = form_data.get("viz_type")
+ if strict_all_form_refs and (
+ viz_type == "mixed_timeseries"
+ or (
+ isinstance(viz_type, str)
+ and (
+ viz_type.startswith("echarts_timeseries") or viz_type == "echarts_area"
+ )
+ )
+ ):
+ if (raw_x_axis := form_data.get("x_axis")) is not None and (
+ error := column_error(raw_x_axis, "form-data x_axis column")
+ ):
+ return error
+ metric_fields = [
+ "metrics",
+ "size",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ ]
+ if viz_type == "mixed_timeseries":
+ metric_fields.extend(
+ [
+ "metrics_b",
+ "size_b",
+ "timeseries_limit_metric_b",
+ "series_limit_metric_b",
+ ]
+ )
+ for field_name in metric_fields:
+ raw_value = form_data.get(field_name)
+ values = raw_value if isinstance(raw_value, list) else [raw_value]
+ for value in values:
+ if value is not None and (
+ error := metric_error(value, f"form-data {field_name} metric")
+ ):
+ return error
+
+ layer_suffixes = ("", "_b") if viz_type == "mixed_timeseries" else ("",)
+ for suffix in layer_suffixes:
+ sort_field = f"x_axis_sort{suffix}"
+ if sort_field not in form_data or form_data.get(sort_field) is None:
+ continue
+ x_axis = form_data.get(f"x_axis{suffix}", form_data.get("x_axis"))
+ allowed_labels: set[str] = set()
+ if x_axis_label := _native_column_label(x_axis):
+ allowed_labels.add(x_axis_label)
+ raw_metrics = form_data.get(f"metrics{suffix}")
+ for metric in raw_metrics if isinstance(raw_metrics, list) else []:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ raw_limit_metric = form_data.get(f"timeseries_limit_metric{suffix}")
+ limit_metrics = (
+ raw_limit_metric
+ if isinstance(raw_limit_metric, list)
+ else [raw_limit_metric]
+ )
+ for metric in limit_metrics:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ sort_value = form_data[sort_field]
+ if not isinstance(sort_value, str) or sort_value not in allowed_labels:
+ return _native_validation_error(sort_field, repr(sort_value)[:200])
+
+ if (
+ strict_all_form_refs
+ and isinstance(viz_type, str)
+ and viz_type.startswith("deck_")
+ ):
+ # Deck layers store most query roles outside common columns/metrics.
+ # Validate every renderer-consumed raw control as well as the generated
+ # QueryObject so a dataset-only rebind cannot hide or discard a stale
+ # tooltip, cross-filter, spatial, path, or metric reference.
+ for spatial_field in ("spatial", "start_spatial", "end_spatial"):
+ spatial = form_data.get(spatial_field)
+ if spatial is None:
+ continue
+ if not isinstance(spatial, dict):
+ return _native_validation_error(
+ f"form-data {spatial_field}", repr(spatial)[:200]
+ )
+ spatial_type = spatial.get("type")
+ if not isinstance(spatial_type, str):
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ role_fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if role_fields is None:
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ for role_field in role_fields:
+ spatial_value = spatial.get(role_field)
+ if spatial_value is None:
+ return _native_validation_error(
+ f"form-data {spatial_field}.{role_field} column", "missing"
+ )
+ if error := column_error(
+ spatial_value, f"form-data {spatial_field}.{role_field} column"
+ ):
+ return error
+
+ for field_name in (
+ "line_column",
+ "geojson",
+ "dimension",
+ "cross_filter_column",
+ ):
+ column_value = form_data.get(field_name)
+ if column_value is not None and (
+ error := column_error(column_value, f"form-data {field_name} column")
+ ):
+ return error
+
+ tooltip_contents = form_data.get("tooltip_contents")
+ if tooltip_contents is not None and not isinstance(tooltip_contents, list):
+ return _native_validation_error(
+ "form-data tooltip_contents", repr(tooltip_contents)[:200]
+ )
+ for index, item in enumerate(tooltip_contents or []):
+ tooltip_value: Any = None
+ if isinstance(item, str):
+ tooltip_value = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ tooltip_value = item.get("column_name")
+ if tooltip_value is not None and (
+ error := column_error(
+ tooltip_value, f"form-data tooltip_contents[{index}] column"
+ )
+ ):
+ return error
+
+ metric_values: list[tuple[str, Any]] = []
+ if viz_type not in {"deck_geojson", "deck_polygon"}:
+ for field_name in ("metrics", "metric", "size"):
+ raw_deck_metrics = form_data.get(field_name)
+ deck_metrics = (
+ raw_deck_metrics
+ if isinstance(raw_deck_metrics, list)
+ else [raw_deck_metrics]
+ )
+ metric_values.extend(
+ (f"form-data {field_name} metric", deck_metric)
+ for deck_metric in deck_metrics
+ if deck_metric is not None
+ )
+ if viz_type == "deck_polygon" and form_data.get("metric") is not None:
+ metric_values.append(("form-data metric metric", form_data.get("metric")))
+ fixed_metric_fields = (
+ ("point_radius_fixed",)
+ if viz_type in {"deck_scatter", "deck_polygon"}
+ else ()
+ ) + (("line_width",) if viz_type == "deck_path" else ())
+ for field_name in fixed_metric_fields:
+ fixed_value = form_data.get(field_name)
+ deck_metric: Any = (
+ fixed_value
+ if (
+ isinstance(fixed_value, str)
+ and fixed_value
+ and viz_type != "deck_polygon"
+ )
+ else None
+ )
+ if isinstance(fixed_value, dict) and fixed_value.get("type") == "metric":
+ deck_metric = fixed_value.get("value")
+ if deck_metric is not None:
+ metric_values.append((f"form-data {field_name} metric", deck_metric))
+ if viz_type == "deck_path" and form_data.get("breakpoint_metric") is not None:
+ metric_values.append(
+ (
+ "form-data breakpoint_metric metric",
+ form_data.get("breakpoint_metric"),
+ )
+ )
+ for role, deck_metric in metric_values:
+ if error := metric_error(deck_metric, role):
+ return error
+
+ for filter_ in form_data.get("adhoc_filters") or []:
+ if not isinstance(filter_, dict) or filter_.get("expressionType") != "SIMPLE":
+ continue
+ if not strict_all_form_refs and _is_inert_adhoc_filter(filter_):
+ continue
+ subject = filter_.get("subject")
+ clause = str(filter_.get("clause") or "WHERE").upper()
+ if clause == "HAVING" and isinstance(subject, str):
+ metric_matches = (
+ [subject]
+ if subject in saved_metrics
+ else [
+ name
+ for name in saved_metrics
+ if name.casefold() == subject.casefold()
+ ]
+ )
+ if len(metric_matches) == 1:
+ continue
+ if subject is not None and (
+ error := column_error(subject, "form-data filter column")
+ ):
+ return error
+ if filter_.get("operator") == "TEMPORAL_RANGE" and isinstance(subject, str):
+ try:
+ temporal = resolve_dataset_column(subject, dataset_context)
+ except ValueError:
+ temporal = None
+ if temporal is not None and not temporal.get("is_temporal", False):
+ return _native_validation_error("temporal filter column", subject)
+
+ # temporal_columns_lookup describes the entire datasource, not selected
+ # roles. The physical form/query column checks validate selected references.
+
+ for query_index, query in enumerate(queries, 1):
+ metric_labels: set[str] = set()
+ for column in query.get("columns") or []:
+ if error := column_error(column, f"query {query_index} column"):
+ return error
+ for column in query.get("series_columns") or []:
+ if error := column_error(column, f"query {query_index} series column"):
+ return error
+ for column in query.get("groupby") or []:
+ if error := column_error(column, f"query {query_index} groupby column"):
+ return error
+ for level in query.get("grouping_sets") or []:
+ for column in level:
+ if error := column_error(
+ column, f"query {query_index} grouping-set column"
+ ):
+ return error
+
+ metrics = query.get("metrics") or []
+ for metric in metrics:
+ if label := _metric_label_for_validation(metric):
+ metric_labels.add(label)
+ if error := metric_error(metric, f"query {query_index} metric"):
+ return error
+
+ granularity = query.get("granularity")
+ if granularity:
+ if error := column_error(granularity, "temporal column"):
+ return error
+ try:
+ temporal = resolve_dataset_column(granularity, dataset_context)
+ except ValueError:
+ temporal = None
+ if (
+ (query.get("extras") or {}).get("time_grain_sqla")
+ and temporal is not None
+ and not temporal.get("is_temporal", False)
+ ):
+ return _native_validation_error("temporal column", granularity)
+
+ for filter_ in query.get("filters") or []:
+ if not isinstance(filter_, dict):
+ return _native_validation_error("filter", repr(filter_)[:200])
+ column = filter_.get("col")
+ if isinstance(column, str) and column in metric_labels:
+ continue
+ if isinstance(column, str) and any(
+ name.casefold() == column.casefold() for name in saved_metrics
+ ):
+ continue
+ if column is not None and (
+ error := column_error(column, f"query {query_index} filter column")
+ ):
+ return error
+
+ for order in query.get("orderby") or []:
+ if not isinstance(order, (list, tuple)) or len(order) != 2:
+ return _native_validation_error("ordering", repr(order)[:200])
+ target = order[0]
+ target_label = _metric_label_for_validation(target)
+ if target in metrics or (target_label and target_label in metric_labels):
+ continue
+ if isinstance(target, str) and target in metric_labels:
+ continue
+ if error := column_error(target, f"query {query_index} ordering column"):
Review Comment:
A saved Table displaying `sum_sales` but sorted by another valid saved metric, `sum_profit`, fails a compatible dataset-only rebind here because the sorter is treated as a physical column unless it is also displayed. Could independent saved/adhoc ordering metrics be validated as metrics, as the native query path supports?
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1338,6 +1340,78 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling. Bullet metric strings use
+JavaScript numeric spellings: underscore separators and non-ASCII digits are
+rejected rather than interpreted as numbers.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates; omitted label controls preserve the saved state.
Review Comment:
Replacing `ranges` while omitting `range_labels` actually clears the saved labels, so following this preservation guarantee can unintentionally remove chart annotations; markers behave the same way. Could this clarify that labels are preserved only when their corresponding value controls are also omitted?
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,441 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ data.setdefault("temporal_column", subject)
Review Comment:
With native temporal filters `OrderDate: No filter` followed by `ShipDate: Last 30 days`, this selects `OrderDate` but takes the range from `ShipDate`, silently filtering the wrong column; multiple active ranges also lose predicates. Could this preserve the subject/range pairs or reject native configurations the typed representation cannot retain?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,544 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
Review Comment:
A zero-valued Bullet measure with `y_axis_format: ".1000000000e"` passes the short-string check but asks the formatter to allocate roughly a billion characters here, before response preflight can reject it; this validation also runs on data reads. Could precision be bounded before evaluating the format so a tiny chart cannot exhaust the worker?
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1894 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
return None
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
Review Comment:
An Arrow list column becomes exact `np.ndarray` cells in pandas, and `df_to_records` preserves them, so even `[1, 2]` now fails with `MalformedQueryResult` before the existing ndarray-to-list serializer runs. Could exact ndarray cells retain bounded array normalization so these dataset/table results remain readable?
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,443 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
- if (
- not isinstance(entry, (list, tuple))
- or len(entry) != 2
- or not isinstance(entry[0], str)
- or not entry[0]
- or not isinstance(entry[1], bool)
- ):
- raise ValueError(
- f"Gantt order_by_cols[{index}] must be [column, ascending_boolean]"
- )
- orderby.append([entry[0], entry[1]])
- columns: list[Any] = []
- seen: set[str] = set()
- for column in (
- start_time,
- end_time,
- category,
- *series_columns,
- *tooltip_columns,
- *(entry[0] for entry in orderby),
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ **(
+ {"isColumnReference": True}
+ if str(form_data.get("datasource", "")).endswith("__semantic_view")
+ else {}
+ ),
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
):
- key = utils_json.dumps(column, sort_keys=True, default=str)
- if key not in seen:
- seen.add(key)
- columns.append(column)
- return columns, list(raw_tooltip_metrics), orderby, series_columns
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
-def extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
- """Return the x_axis column name from form_data, or None if not set."""
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
x_axis = form_data.get("x_axis")
- if isinstance(x_axis, str) and x_axis:
- return x_axis
- if isinstance(x_axis, dict):
- col_name = x_axis.get("column_name")
- return col_name if isinstance(col_name, str) and col_name else None
- return None
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
+ **(
+ {"fill_time_range": True}
+ if form_data.get("resample_fill_time_range")
+ else {}
+ ),
+ },
+ }
+ )
+ rolling_type = form_data.get("rolling_type")
+ rolling_columns = (
+ [*offset_map.values(), *offset_map.keys()] if comparison else metric_labels
+ )
+ if rolling_type == "cumsum":
+ chain.append(
+ {
+ "operation": "cum",
+ "options": {
+ "operator": "sum",
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ elif rolling_type in {"sum", "mean", "std"}:
+ chain.append(
+ {
+ "operation": "rolling",
+ "options": {
+ "rolling_type": rolling_type,
+ "window": int(form_data.get("rolling_periods") or 1),
+ "min_periods": int(form_data.get("min_periods") or 0),
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ comparison_type = form_data.get("comparison_type")
+ if comparison and comparison_type != "values":
+ chain.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": list(offset_map.values()),
+ "compare_columns": list(offset_map.keys()),
+ "compare_type": comparison_type,
+ "drop_original_columns": True,
+ },
+ }
+ )
+ if complete_timeseries_contract and form_data.get("contributionMode"):
+ chain.append(
+ {
+ "operation": "contribution",
+ "options": {
+ "orientation": form_data["contributionMode"],
+ "time_shifts": offsets if comparison else [],
+ },
+ }
+ )
+ if comparison:
+ rename: dict[str, str | None] = {}
+ for shifted, metric in offset_map.items():
+ offset = next(
+ (item for item in offsets if shifted.endswith(f"__{item}")), None
+ )
+ source = (
+ shifted
+ if comparison_type == "values"
+ else f"{comparison_type}__{metric}__{shifted}"
+ )
+ rename[source] = f"{metric}, {offset}" if len(metrics) > 1 else offset
+ if rename:
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {"columns": rename, "level": 0, "inplace": True},
+ }
+ )
+ elif (
+ x_label
+ and len(metrics) == 1
+ and (series_labels or len(offsets) > 1)
+ and form_data.get("truncate_metric") is not None
Review Comment:
After clearing time shifts while retaining `comparison_type: "difference"` and `truncate_metric: true`, a grouped single-metric query drops `Revenue, US` to `US` here, changing cached data/export column names from Explore. Could this retain the native `renameOperator` exclusion for `difference`, `percentage`, and `ratio` even when no shifts are active?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4178954268
##########
superset/mcp_service/chart/compile.py:
##########
@@ -282,6 +329,439 @@ def _validate_adhoc_filter_columns(
)
+def _native_validation_error(role: str, reference: str) -> ChartGenerationError:
+ """Build a fail-closed error for an incompatible native chart reference."""
+ return ChartGenerationError(
+ error_type="invalid_native_chart_reference",
+ message=f"Native chart {role} {reference!r} is incompatible with the dataset",
+ details=(
+ "The rebound form data must retain its exact query roles on the target "
+ "dataset; no column or saved-metric reference may be guessed or dropped."
+ ),
+ suggestions=[
+ "Choose a target dataset with a compatible schema",
+ "Provide a complete typed chart config using target-dataset fields",
+ ],
+ error_code="CHART_VALIDATION_FAILED",
+ )
+
+
+def _native_column_name(value: Any) -> str | None:
+ """Extract a physical QueryFormColumn reference, or None for SQL columns."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ reference = value.get("sqlExpression")
+ if value.get("isColumnReference") is True and isinstance(reference, str):
+ return reference or None
+ return None
+ name = value.get("column_name") or value.get("columnName")
+ return name if isinstance(name, str) and name else None
+
+
+def _native_column_label(value: Any) -> str | None:
+ """Return the frontend label for a native column without custom hooks."""
+ if isinstance(value, str):
+ return value
+ if not isinstance(value, dict):
+ return None
+ for key in ("label", "sqlExpression", "column_name", "columnName"):
+ candidate = value.get(key)
+ if isinstance(candidate, str) and candidate:
+ return candidate
+ return None
+
+
+def _native_metric_ref(value: Any) -> tuple[str, str] | None:
+ """Return ``(saved_metric|column, name)`` for a native query metric."""
+ if isinstance(value, str):
+ return "saved_metric", value
+ if not isinstance(value, dict):
+ return None
+ if value.get("expressionType") == "SQL":
+ return None
+ if value.get("expressionType") != "SIMPLE":
+ # Match QueryObject's guarded legacy saved-metric normalization.
+ if not ({"sqlExpression", "aggregate", "column"} & value.keys()):
+ label = value.get("label")
+ if isinstance(label, str) and label:
+ return "saved_metric", label
+ return None
+ column = value.get("column")
+ name = (
+ column.get("column_name") or column.get("columnName")
+ if isinstance(column, dict)
+ else None
+ )
+ return ("column", name) if isinstance(name, str) and name else None
+
+
+def _native_reference_error( # noqa: C901
+ form_data: Dict[str, Any],
+ dataset_context: DatasetContext,
+ dataset_id: int,
+ *,
+ strict_all_form_refs: bool,
+) -> ChartGenerationError | None:
+ """Validate the canonical native QueryObjects against a rebound dataset."""
+ from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+ )
+
+ try:
+ queries = build_query_dicts_from_form_data(
+ deepcopy(form_data), dataset_id, "table"
+ )
+ except (KeyError, TypeError, ValueError) as ex:
+ return _native_validation_error("query contract", safe_exception_message(ex))
+
+ saved_metrics = [item["name"] for item in dataset_context.available_metrics]
+
+ def column_error(value: Any, role: str) -> ChartGenerationError | None:
+ name = _native_column_name(value)
+ if name is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ try:
+ if resolve_dataset_column(name, dataset_context) is not None:
+ return None
+ except ValueError:
+ pass
+ return _native_validation_error(role, name)
+
+ def metric_error(value: Any, role: str) -> ChartGenerationError | None:
+ """Validate one raw or generated metric reference against the target."""
+ ref = _native_metric_ref(value)
+ if ref is None:
+ if isinstance(value, dict) and value.get("expressionType") == "SQL":
+ return None
+ return _native_validation_error(role, repr(value)[:200])
+ kind, name = ref
+ if kind == "saved_metric":
+ # Native lookup selects an exact name unambiguously; only a
+ # case-folded reference has to be unique.
+ matches = (
+ [name]
+ if name in saved_metrics
+ else [
+ item for item in saved_metrics if item.casefold() == name.casefold()
+ ]
+ )
+ if len(set(matches)) != 1:
+ saved_role = f"{role.removesuffix(' metric')} saved metric"
+ return _native_validation_error(saved_role, name)
+ return None
+ return column_error(name, f"{role} column")
+
+ # Dataset-only rebind has no typed config to expose these native plugin
+ # roles. Validate the raw controls independently: some are consumed only
+ # while building ordering/post-processing and therefore may be absent from
+ # the final QueryObject (notably an explicit ordering can hide a ranking
+ # metric). Primary and secondary Mixed layers are deliberately separate.
+ viz_type = form_data.get("viz_type")
+ if strict_all_form_refs and (
+ viz_type == "mixed_timeseries"
+ or (
+ isinstance(viz_type, str)
+ and (
+ viz_type.startswith("echarts_timeseries") or viz_type == "echarts_area"
+ )
+ )
+ ):
+ if (raw_x_axis := form_data.get("x_axis")) is not None and (
+ error := column_error(raw_x_axis, "form-data x_axis column")
+ ):
+ return error
+ metric_fields = [
+ "metrics",
+ "size",
+ "timeseries_limit_metric",
+ "series_limit_metric",
+ ]
+ if viz_type == "mixed_timeseries":
+ metric_fields.extend(
+ [
+ "metrics_b",
+ "size_b",
+ "timeseries_limit_metric_b",
+ "series_limit_metric_b",
+ ]
+ )
+ for field_name in metric_fields:
+ raw_value = form_data.get(field_name)
+ values = raw_value if isinstance(raw_value, list) else [raw_value]
+ for value in values:
+ if value is not None and (
+ error := metric_error(value, f"form-data {field_name} metric")
+ ):
+ return error
+
+ layer_suffixes = ("", "_b") if viz_type == "mixed_timeseries" else ("",)
+ for suffix in layer_suffixes:
+ sort_field = f"x_axis_sort{suffix}"
+ if sort_field not in form_data or form_data.get(sort_field) is None:
+ continue
+ x_axis = form_data.get(f"x_axis{suffix}", form_data.get("x_axis"))
+ allowed_labels: set[str] = set()
+ if x_axis_label := _native_column_label(x_axis):
+ allowed_labels.add(x_axis_label)
+ raw_metrics = form_data.get(f"metrics{suffix}")
+ for metric in raw_metrics if isinstance(raw_metrics, list) else []:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ raw_limit_metric = form_data.get(f"timeseries_limit_metric{suffix}")
+ limit_metrics = (
+ raw_limit_metric
+ if isinstance(raw_limit_metric, list)
+ else [raw_limit_metric]
+ )
+ for metric in limit_metrics:
+ if label := _metric_label_for_validation(metric):
+ allowed_labels.add(label)
+ sort_value = form_data[sort_field]
+ if not isinstance(sort_value, str) or sort_value not in allowed_labels:
+ return _native_validation_error(sort_field, repr(sort_value)[:200])
+
+ if (
+ strict_all_form_refs
+ and isinstance(viz_type, str)
+ and viz_type.startswith("deck_")
+ ):
+ # Deck layers store most query roles outside common columns/metrics.
+ # Validate every renderer-consumed raw control as well as the generated
+ # QueryObject so a dataset-only rebind cannot hide or discard a stale
+ # tooltip, cross-filter, spatial, path, or metric reference.
+ for spatial_field in ("spatial", "start_spatial", "end_spatial"):
+ spatial = form_data.get(spatial_field)
+ if spatial is None:
+ continue
+ if not isinstance(spatial, dict):
+ return _native_validation_error(
+ f"form-data {spatial_field}", repr(spatial)[:200]
+ )
+ spatial_type = spatial.get("type")
+ if not isinstance(spatial_type, str):
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ role_fields = {
+ "latlong": ("lonCol", "latCol"),
+ "delimited": ("lonlatCol",),
+ "geohash": ("geohashCol",),
+ }.get(spatial_type)
+ if role_fields is None:
+ return _native_validation_error(
+ f"form-data {spatial_field} type", repr(spatial_type)[:200]
+ )
+ for role_field in role_fields:
+ spatial_value = spatial.get(role_field)
+ if spatial_value is None:
+ return _native_validation_error(
+ f"form-data {spatial_field}.{role_field} column", "missing"
+ )
+ if error := column_error(
+ spatial_value, f"form-data {spatial_field}.{role_field} column"
+ ):
+ return error
+
+ for field_name in (
+ "line_column",
+ "geojson",
+ "dimension",
+ "cross_filter_column",
+ ):
+ column_value = form_data.get(field_name)
+ if column_value is not None and (
+ error := column_error(column_value, f"form-data {field_name} column")
+ ):
+ return error
+
+ tooltip_contents = form_data.get("tooltip_contents")
+ if tooltip_contents is not None and not isinstance(tooltip_contents, list):
+ return _native_validation_error(
+ "form-data tooltip_contents", repr(tooltip_contents)[:200]
+ )
+ for index, item in enumerate(tooltip_contents or []):
+ tooltip_value: Any = None
+ if isinstance(item, str):
+ tooltip_value = item
+ elif isinstance(item, dict) and item.get("item_type") == "column":
+ tooltip_value = item.get("column_name")
+ if tooltip_value is not None and (
+ error := column_error(
+ tooltip_value, f"form-data tooltip_contents[{index}] column"
+ )
+ ):
+ return error
+
+ metric_values: list[tuple[str, Any]] = []
+ if viz_type not in {"deck_geojson", "deck_polygon"}:
+ for field_name in ("metrics", "metric", "size"):
+ raw_deck_metrics = form_data.get(field_name)
+ deck_metrics = (
+ raw_deck_metrics
+ if isinstance(raw_deck_metrics, list)
+ else [raw_deck_metrics]
+ )
+ metric_values.extend(
+ (f"form-data {field_name} metric", deck_metric)
+ for deck_metric in deck_metrics
+ if deck_metric is not None
+ )
+ if viz_type == "deck_polygon" and form_data.get("metric") is not None:
+ metric_values.append(("form-data metric metric", form_data.get("metric")))
+ fixed_metric_fields = (
+ ("point_radius_fixed",)
+ if viz_type in {"deck_scatter", "deck_polygon"}
+ else ()
+ ) + (("line_width",) if viz_type == "deck_path" else ())
+ for field_name in fixed_metric_fields:
+ fixed_value = form_data.get(field_name)
+ deck_metric: Any = (
+ fixed_value
+ if (
+ isinstance(fixed_value, str)
+ and fixed_value
+ and viz_type != "deck_polygon"
+ )
+ else None
+ )
+ if isinstance(fixed_value, dict) and fixed_value.get("type") == "metric":
+ deck_metric = fixed_value.get("value")
+ if deck_metric is not None:
+ metric_values.append((f"form-data {field_name} metric", deck_metric))
+ if viz_type == "deck_path" and form_data.get("breakpoint_metric") is not None:
+ metric_values.append(
+ (
+ "form-data breakpoint_metric metric",
+ form_data.get("breakpoint_metric"),
+ )
+ )
+ for role, deck_metric in metric_values:
+ if error := metric_error(deck_metric, role):
+ return error
+
+ for filter_ in form_data.get("adhoc_filters") or []:
+ if not isinstance(filter_, dict) or filter_.get("expressionType") != "SIMPLE":
+ continue
+ if not strict_all_form_refs and _is_inert_adhoc_filter(filter_):
+ continue
+ subject = filter_.get("subject")
+ clause = str(filter_.get("clause") or "WHERE").upper()
+ if clause == "HAVING" and isinstance(subject, str):
+ metric_matches = (
+ [subject]
+ if subject in saved_metrics
+ else [
+ name
+ for name in saved_metrics
+ if name.casefold() == subject.casefold()
+ ]
+ )
+ if len(metric_matches) == 1:
+ continue
+ if subject is not None and (
+ error := column_error(subject, "form-data filter column")
+ ):
+ return error
+ if filter_.get("operator") == "TEMPORAL_RANGE" and isinstance(subject, str):
+ try:
+ temporal = resolve_dataset_column(subject, dataset_context)
+ except ValueError:
+ temporal = None
+ if temporal is not None and not temporal.get("is_temporal", False):
+ return _native_validation_error("temporal filter column", subject)
+
+ # temporal_columns_lookup describes the entire datasource, not selected
+ # roles. The physical form/query column checks validate selected references.
+
+ for query_index, query in enumerate(queries, 1):
+ metric_labels: set[str] = set()
+ for column in query.get("columns") or []:
+ if error := column_error(column, f"query {query_index} column"):
+ return error
+ for column in query.get("series_columns") or []:
+ if error := column_error(column, f"query {query_index} series column"):
+ return error
+ for column in query.get("groupby") or []:
+ if error := column_error(column, f"query {query_index} groupby column"):
+ return error
+ for level in query.get("grouping_sets") or []:
+ for column in level:
+ if error := column_error(
+ column, f"query {query_index} grouping-set column"
+ ):
+ return error
+
+ metrics = query.get("metrics") or []
+ for metric in metrics:
+ if label := _metric_label_for_validation(metric):
+ metric_labels.add(label)
+ if error := metric_error(metric, f"query {query_index} metric"):
+ return error
+
+ granularity = query.get("granularity")
+ if granularity:
+ if error := column_error(granularity, "temporal column"):
+ return error
+ try:
+ temporal = resolve_dataset_column(granularity, dataset_context)
+ except ValueError:
+ temporal = None
+ if (
+ (query.get("extras") or {}).get("time_grain_sqla")
+ and temporal is not None
+ and not temporal.get("is_temporal", False)
+ ):
+ return _native_validation_error("temporal column", granularity)
+
+ for filter_ in query.get("filters") or []:
+ if not isinstance(filter_, dict):
+ return _native_validation_error("filter", repr(filter_)[:200])
+ column = filter_.get("col")
+ if isinstance(column, str) and column in metric_labels:
+ continue
+ if isinstance(column, str) and any(
+ name.casefold() == column.casefold() for name in saved_metrics
+ ):
+ continue
+ if column is not None and (
+ error := column_error(column, f"query {query_index} filter column")
+ ):
+ return error
+
+ for order in query.get("orderby") or []:
+ if not isinstance(order, (list, tuple)) or len(order) != 2:
+ return _native_validation_error("ordering", repr(order)[:200])
+ target = order[0]
+ target_label = _metric_label_for_validation(target)
+ if target in metrics or (target_label and target_label in metric_labels):
+ continue
+ if isinstance(target, str) and target in metric_labels:
+ continue
+ if error := column_error(target, f"query {query_index} ordering column"):
Review Comment:
Introduced by this PR's dataset-only native rebind validator; master does not have this validator. Fixed in 02453fbea4bcb29fe20087f2d08d1ba333f3bbf1. Independent saved ordering metrics (including case-folded references) and SIMPLE adhoc ordering metrics are validated as metrics even when not displayed. Regression tests cover compatible saved/adhoc sorters and an incompatible sorter column.
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -2713,6 +2820,441 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
+ if not isinstance(target, str) or not target:
+ raise ValueError(f"order_by[{index}] needs a column or metric label")
+ if not isinstance(ascending, bool):
+ raise ValueError(f"order_by[{index}] ascending value must be boolean")
+ result.append({"column": target, "ascending": ascending})
+ return result
+
+ @staticmethod
+ def _adapt_native_filters(data: dict[str, Any]) -> None: # noqa: C901
+ if "adhoc_filters" not in data:
+ return
+ if "filters" in data:
+ raise ValueError("Use either filters or native adhoc_filters, not both")
+ raw_filters = data.pop("adhoc_filters")
+ if not isinstance(raw_filters, list):
+ raise ValueError("adhoc_filters must be an array")
+ filters: list[dict[str, Any]] = []
+ for index, raw_filter in enumerate(raw_filters):
+ if not isinstance(raw_filter, dict):
+ raise ValueError(f"adhoc_filters[{index}] must be an object")
+ if raw_filter.get("expressionType") != "SIMPLE":
+ raise ValueError(
+ f"adhoc_filters[{index}] must use expressionType='SIMPLE'"
+ )
+ if raw_filter.get("clause") not in (None, "WHERE"):
+ raise ValueError(f"adhoc_filters[{index}] must use clause='WHERE'")
+ subject = raw_filter.get("subject")
+ operator = raw_filter.get("operator")
+ comparator = raw_filter.get("comparator")
+ if operator == "TEMPORAL_RANGE":
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ f"adhoc_filters[{index}] temporal filter needs subject"
+ )
+ data.setdefault("temporal_column", subject)
Review Comment:
Introduced by this PR's Bullet native-filter adapter; master has no Bullet typed adapter. Fixed in 02453fbea4bcb29fe20087f2d08d1ba333f3bbf1. The active temporal filter retains its own subject/range pair regardless of placeholder order. Multiple active native temporal filters and conflicts with explicit temporal_column/time_range are rejected instead of silently dropping or moving predicates. Regression tests cover both filter orders and the conflicting configurations.
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1894 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
return None
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
Review Comment:
Introduced by this PR's strict row normalization; master does not reject these cells through this normalization path. Fixed in 02453fbea4bcb29fe20087f2d08d1ba333f3bbf1. Exact ndarray cells are expanded one bounded dimension at a time and normalized to JSON arrays. Tests use real Arrow list and nested-list columns through pandas and df_to_records, and retain width, depth, cycle, subclass, cumulative work, and exact JSON-byte boundary checks.
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +414,544 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _javascript_numeric_string(value: str) -> float:
+ """Parse a nonempty trimmed string using JavaScript Number's grammar."""
+ if re.fullmatch(r"0[xX][0-9a-fA-F]+|0[bB][01]+|0[oO][0-7]+", value):
+ return float(int(value, 0))
+ if re.fullmatch(
+ r"[+-]?(?:Infinity|(?:[0-9]+(?:\.[0-9]*)?|\.[0-9]+)(?:[eE][+-]?[0-9]+)?)",
+ value,
+ ):
+ return float(value)
+ raise ValueError("Invalid JavaScript number spelling")
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip(_JAVASCRIPT_WHITESPACE)
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = _javascript_numeric_string(stripped)
+ except (ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
Review Comment:
The generic formatter predates this PR, but the Bullet validation/data-read invocation without a pre-format precision bound is introduced here. Fixed in 02453fbea4bcb29fe20087f2d08d1ba333f3bbf1 in the PR-added Bullet validation path: parsed precision above 20 is rejected before any formatter invocation. Regression tests verify the billion-digit specifiers never call the formatter, including for a zero measure, and that precision 20 remains supported.
##########
superset/mcp_service/chart/chart_helpers.py:
##########
@@ -615,75 +1112,443 @@ def require_column(value: Any, field_name: str) -> Any:
start_time = require_column(form_data.get("start_time"), "start_time")
end_time = require_column(form_data.get("end_time"), "end_time")
category = require_column(form_data.get("y_axis"), "y_axis")
-
raw_series = form_data.get("series")
series_columns = (
[require_column(raw_series, "series")] if raw_series is not None else []
)
-
- raw_tooltip_columns = form_data.get("tooltip_columns") or []
- raw_tooltip_metrics = form_data.get("tooltip_metrics") or []
- if not isinstance(raw_tooltip_columns, list) or len(raw_tooltip_columns) > 50:
+ raw_tooltips = form_data.get("tooltip_columns") or []
+ raw_metrics = form_data.get("tooltip_metrics") or []
+ if not isinstance(raw_tooltips, list) or len(raw_tooltips) > 50:
raise ValueError("Gantt tooltip_columns must contain at most 50 entries")
- if not isinstance(raw_tooltip_metrics, list) or len(raw_tooltip_metrics) > 50:
+ if not isinstance(raw_metrics, list) or len(raw_metrics) > 50:
raise ValueError("Gantt tooltip_metrics must contain at most 50 entries")
tooltip_columns = [
require_column(column, f"tooltip_columns[{index}]")
- for index, column in enumerate(raw_tooltip_columns)
+ for index, column in enumerate(raw_tooltips)
]
+ orderby = _parse_orderby(form_data.get("order_by_cols"))
+ columns = _dedupe_query_fields(
+ [
+ start_time,
+ end_time,
+ category,
+ *series_columns,
+ *tooltip_columns,
+ *(item[0] for item in orderby),
+ ],
+ _column_label,
+ )
+ return columns, list(raw_metrics), orderby, series_columns
- raw_order = form_data.get("order_by_cols") or []
- if not isinstance(raw_order, list) or len(raw_order) > 100:
- raise ValueError("Gantt order_by_cols must contain at most 100 entries")
- orderby: list[list[Any]] = []
- for index, entry in enumerate(raw_order):
- if isinstance(entry, str):
- if len(entry) > 1000:
- raise ValueError(f"Gantt order_by_cols[{index}] is too long")
- try:
- entry = utils_json.loads(entry)
- except (TypeError, ValueError) as ex:
- raise ValueError(
- f"Gantt order_by_cols[{index}] is not valid JSON"
- ) from ex
- if (
- not isinstance(entry, (list, tuple))
- or len(entry) != 2
- or not isinstance(entry[0], str)
- or not entry[0]
- or not isinstance(entry[1], bool)
- ):
- raise ValueError(
- f"Gantt order_by_cols[{index}] must be [column, ascending_boolean]"
- )
- orderby.append([entry[0], entry[1]])
- columns: list[Any] = []
- seen: set[str] = set()
- for column in (
- start_time,
- end_time,
- category,
- *series_columns,
- *tooltip_columns,
- *(entry[0] for entry in orderby),
+def _table_time_offsets(form_data: dict[str, Any], query: dict[str, Any]) -> list[Any]:
+ """Resolve the Table plugin's custom/inherit comparison offsets."""
+ if not _time_comparison(form_data, query.get("metrics") or []):
+ return []
+ offsets: list[Any] = []
+ for offset in _as_list(form_data.get("time_compare")):
+ if offset == "custom":
+ offset = form_data.get("start_date_offset")
+ elif offset == "inherit":
+ offset = "inherit"
+ if offset is not None and offset not in offsets:
+ offsets.append(offset)
+ extra = form_data.get("extra_form_data")
+ if isinstance(extra, dict):
+ offset = extra.get("time_compare")
+ if offset is not None and offset not in offsets:
+ offsets = [offset]
+ return offsets
+
+
+def _table_totals_metrics(metrics: list[Any], aggregate: Any) -> list[Any]:
+ """Mirror ``getTotalsMetrics`` for Table summary queries."""
+ if aggregate not in {"SUM", "AVG"}:
+ return metrics
+ result: list[Any] = []
+ for metric in metrics:
+ if isinstance(metric, dict) and metric.get("expressionType") == "SIMPLE":
+ result.append({**metric, "aggregate": aggregate})
+ else:
+ result.append(metric)
+ return result
+
+
+def _temporal_column(column: Any, form_data: dict[str, Any]) -> Any:
+ """Apply the frontend BASE_AXIS wrapper for a physical temporal column."""
+ if not isinstance(column, str) or not form_data.get("time_grain_sqla"):
+ return column
+ lookup = form_data.get("temporal_columns_lookup")
+ if not isinstance(lookup, dict) or not lookup.get(column):
+ return column
+ return {
+ "timeGrain": form_data["time_grain_sqla"],
+ "columnType": "BASE_AXIS",
+ "sqlExpression": column,
+ "label": column,
+ "expressionType": "SQL",
+ **(
+ {"isColumnReference": True}
+ if str(form_data.get("datasource", "")).endswith("__semantic_view")
+ else {}
+ ),
+ }
+
+
+def _normalize_orderby(query: dict[str, Any]) -> None:
+ """Mirror ``normalizeOrderBy`` without dropping independent mixed state."""
+ orderby = query.get("orderby")
+ if (
+ isinstance(orderby, list)
+ and orderby
+ and isinstance(orderby[0], (list, tuple))
+ and len(orderby[0]) == 2
+ and orderby[0][0]
+ and isinstance(orderby[0][1], bool)
):
- key = utils_json.dumps(column, sort_keys=True, default=str)
- if key not in seen:
- seen.add(key)
- columns.append(column)
- return columns, list(raw_tooltip_metrics), orderby, series_columns
+ return
+ query.pop("orderby", None)
+ target = query.get("series_limit_metric") or query.get("legacy_order_by")
+ if target is None:
+ metrics = query.get("metrics") or []
+ target = metrics[0] if metrics else None
+ if target is not None:
+ query["orderby"] = [[target, not query.get("order_desc", True)]]
+
+
+def _time_comparison(form_data: dict[str, Any], metrics: list[Any]) -> bool:
+ return bool(
+ metrics
+ and _as_list(form_data.get("time_compare"))
+ and form_data.get("comparison_type")
+ in {"values", "difference", "percentage", "ratio"}
+ )
-def extract_x_axis_col(form_data: dict[str, Any]) -> str | None:
- """Return the x_axis column name from form_data, or None if not set."""
+def _timeseries_post_processing( # noqa: C901
+ form_data: dict[str, Any],
+ query: dict[str, Any],
+ *,
+ operator_metrics: list[Any] | None = None,
+ complete_timeseries_contract: bool = False,
+) -> list[dict[str, Any]]:
+ """Build the frontend Mixed/Timeseries post-processing contract.
+
+ Timeseries passes its pre-extra-metric QueryObject to every operator, while
+ adding ``extractExtraMetrics`` only to its final query and normal pivot.
+ Mixed passes each layer QueryObject and implements the smaller operator set
+ in its own frontend builder.
+ """
+ metrics = (
+ list(operator_metrics)
+ if operator_metrics is not None
+ else list(query.get("metrics") or [])
+ )
+ metric_labels = [label for metric in metrics if (label := _metric_label(metric))]
x_axis = form_data.get("x_axis")
- if isinstance(x_axis, str) and x_axis:
- return x_axis
- if isinstance(x_axis, dict):
- col_name = x_axis.get("column_name")
- return col_name if isinstance(col_name, str) and col_name else None
- return None
+ x_label = (
+ _column_label(x_axis)
+ if x_axis
+ else ("__timestamp" if form_data.get("granularity_sqla") else None)
+ )
+ series = _query_series_columns(query)
+ series_labels = [label for column in series if (label := _column_label(column))]
+ offsets = _as_list(form_data.get("time_compare"))
+ comparison = _time_comparison(form_data, metrics)
+ offset_map = {
+ f"{metric}__{offset}": metric for metric in metric_labels for offset in offsets
+ }
+ pivot_metrics = (
+ [*offset_map.values(), *offset_map.keys()]
+ if comparison
+ else [
+ *metric_labels,
+ *(
+ [
+ label
+ for metric in _timeseries_extra_metrics(form_data)
+ if (label := _metric_label(metric))
+ ]
+ if complete_timeseries_contract
+ else []
+ ),
+ ]
+ )
+ chain: list[dict[str, Any] | None] = []
+ if x_label and pivot_metrics:
+ chain.append(
+ {
+ "operation": "pivot",
+ "options": {
+ "index": [x_label],
+ "columns": series_labels,
+ "aggregates": {
+ metric: {"operator": "mean"} for metric in pivot_metrics
+ },
+ "drop_missing_columns": not form_data.get(
+ "show_empty_columns", False
+ ),
+ },
+ }
+ )
+ method = form_data.get("resample_method")
+ rule = form_data.get("resample_rule")
+ if method and rule:
+ zero_fill = method == "zerofill"
+ chain.append(
+ {
+ "operation": "resample",
+ "options": {
+ "method": "asfreq" if zero_fill else method,
+ "rule": rule,
+ "fill_value": 0 if zero_fill else None,
+ **(
+ {"fill_time_range": True}
+ if form_data.get("resample_fill_time_range")
+ else {}
+ ),
+ },
+ }
+ )
+ rolling_type = form_data.get("rolling_type")
+ rolling_columns = (
+ [*offset_map.values(), *offset_map.keys()] if comparison else metric_labels
+ )
+ if rolling_type == "cumsum":
+ chain.append(
+ {
+ "operation": "cum",
+ "options": {
+ "operator": "sum",
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ elif rolling_type in {"sum", "mean", "std"}:
+ chain.append(
+ {
+ "operation": "rolling",
+ "options": {
+ "rolling_type": rolling_type,
+ "window": int(form_data.get("rolling_periods") or 1),
+ "min_periods": int(form_data.get("min_periods") or 0),
+ "columns": {column: column for column in rolling_columns},
+ },
+ }
+ )
+ comparison_type = form_data.get("comparison_type")
+ if comparison and comparison_type != "values":
+ chain.append(
+ {
+ "operation": "compare",
+ "options": {
+ "source_columns": list(offset_map.values()),
+ "compare_columns": list(offset_map.keys()),
+ "compare_type": comparison_type,
+ "drop_original_columns": True,
+ },
+ }
+ )
+ if complete_timeseries_contract and form_data.get("contributionMode"):
+ chain.append(
+ {
+ "operation": "contribution",
+ "options": {
+ "orientation": form_data["contributionMode"],
+ "time_shifts": offsets if comparison else [],
+ },
+ }
+ )
+ if comparison:
+ rename: dict[str, str | None] = {}
+ for shifted, metric in offset_map.items():
+ offset = next(
+ (item for item in offsets if shifted.endswith(f"__{item}")), None
+ )
+ source = (
+ shifted
+ if comparison_type == "values"
+ else f"{comparison_type}__{metric}__{shifted}"
+ )
+ rename[source] = f"{metric}, {offset}" if len(metrics) > 1 else offset
+ if rename:
+ chain.append(
+ {
+ "operation": "rename",
+ "options": {"columns": rename, "level": 0, "inplace": True},
+ }
+ )
+ elif (
+ x_label
+ and len(metrics) == 1
+ and (series_labels or len(offsets) > 1)
+ and form_data.get("truncate_metric") is not None
Review Comment:
Introduced by this PR's server-side timeseries post-processing port. Fixed in 02453fbea4bcb29fe20087f2d08d1ba333f3bbf1. Metric truncation retains the native renameOperator exclusions for difference, percentage, and ratio even after time shifts are cleared. Regression coverage checks all three excluded modes, plus values and the default mode as positive truncation controls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4178955004
##########
docs/admin_docs/configuration/mcp-server.mdx:
##########
@@ -1338,6 +1340,78 @@ Disabling a plugin only stops new charts of that type from being created. Existi
- **[Security](/developer-docs/extensions/security)** -- Security best practices for extensions
- **[Deployment](/developer-docs/extensions/deployment)** -- Package and deploy Superset extensions
+## Bullet chart compatibility
+
+The MCP Bullet plugin uses `chart_type: "bullet"` and the native ECharts
+`viz_type: "bullet"`. Its optional `dimensions` hierarchy maps to `groupby`.
+Omit `dimensions` (or use `null`) to create a single-metric Bullet without a
+breakdown. On updates, omission or `null` preserves the saved hierarchy; use
+`dimensions: []` to clear it explicitly. An `order_by` update can reference the
+saved dimensions without resending them; unknown targets are rejected after
+resolving the saved hierarchy. When replacing the dataset, saved-chart and
+cached-preview updates retain an omitted hierarchy only if its columns resolve
+in the replacement dataset; incompatible inherited roles and temporal-filter
+provenance are discarded.
+
+Dimension and metric output names are case-sensitive: quoted physical columns
+such as `Region` and `region` remain distinct. Reference lookup prefers exact
+names and uses case-insensitive matching only when there is a single candidate;
+ambiguous references require the exact spelling. Bullet metric strings use
+JavaScript numeric spellings: underscore separators and non-ASCII digits are
+rejected rather than interpreted as numbers.
+
+Range, marker, and marker-line label lists may be shorter than their value lists.
+As in Explore, missing or empty range labels are not displayed, and missing or
+empty marker labels use the formatted numeric value. Extra labels have no value
+to annotate and are ignored. These rules also apply to saved-chart previews and
+updates; omitted label controls preserve the saved state.
Review Comment:
Introduced by this PR's documentation; clearing omitted labels when replacing their value controls is intentional. Clarified in 02453fbea4bcb29fe20087f2d08d1ba333f3bbf1: labels are preserved only when the corresponding ranges, markers, or marker_lines control is also omitted; resend labels with replacement values to retain annotations. Extended the regression to verify both preservation and clearing for all three control pairs. Validation: 3,637 chart unit tests passed, 3 skipped; staged pre-commit passed, including mypy and pylint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4189976978
##########
superset/mcp_service/chart/query_result.py:
##########
@@ -18,90 +18,1983 @@
"""Helpers for interpreting ChartDataCommand result envelopes."""
import math
-from collections.abc import Mapping
-from decimal import Decimal
+import re
+import time as system_time
+from bisect import bisect_right
+from collections.abc import Mapping, Sequence
+from dataclasses import dataclass
+from datetime import date, datetime, time, timedelta, timezone
+from decimal import Decimal, InvalidOperation
+from enum import Enum
from numbers import Real
+from types import MappingProxyType
from typing import Any, cast
+from uuid import UUID
+from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
+
+import numpy as np
+import pandas as pd
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.tz.tz import _ttinfo as dateutil_ttinfo
+from dateutil.zoneinfo import tzfile as dateutil_zoneinfo_tzfile
+from pydantic import BaseModel
+from pydantic_core import to_json
from superset.mcp_service.chart.schemas import ChartError
+from superset.mcp_service.utils.serialization import decode_binary
+from superset.utils.core import GenericDataType
+from superset.utils.dates import datetime_to_epoch, EPOCH
FAILED_QUERY_STATUSES = frozenset(
{"error", "failed", "stopped", "timed_out", "cancelled", "canceled"}
)
+_ERROR_KEYS = ("error", "error_message", "message", "detail")
+_MAX_ERROR_DEPTH = 32
+_MAX_ERROR_ITEMS = 256
+_MAX_SEQUENCE_ITEMS = 64
+_MAX_ERROR_PARTS = 3
+_MAX_ERROR_BYTES = 2000
+_MAX_INTEGER_DIGITS = 1000
+_MAX_QUERY_COUNT = 64
+_MAX_QUERY_COLUMNS = 4096
+_MAX_COLUMN_NAME_BYTES = 4096
+_MAX_ROW_CONTAINER_DEPTH = 32
+_MAX_ROW_CONTAINER_ITEMS = 4096
+_MAX_CACHE_STRING_BYTES = 4096
+_MAX_RESULT_ROW_COUNT = (1 << 63) - 1
-def _query_error_text(value: Any) -> str | None:
- """Convert a bounded query error payload into a useful message."""
- if value is None or value is False:
+# Chart results are routinely much larger than an MCP response should return, but
+# legitimate exports and high-cardinality chart queries still need useful room.
+# Each query may return Superset's configured 50k ROW_LIMIT. The aggregate row
+# budget admits both legs of Big Number raw/trend and Mixed Timeseries results
+# at that limit, while the value budget admits twenty scalar columns on both
+# legs (plus their row containers). The complete compact JSON projection is
+# capped at 16 MiB, including scalar tokens, escaping, keys, and syntax. Metadata
+# profiling has a separate row-by-column work budget in ``response_utils`` so
+# wide sparse results cannot turn bounded validation into an unbounded scan.
+# Individual source-result cell strings are capped at 64 KiB and object keys at
+# 4 KiB. Derived strings in a final Pydantic response have no per-cell cap; the
+# complete compact response remains subject to the 16 MiB aggregate budget.
+# Query metadata has its own 1 MiB aggregate budget so SQL and cache metadata
+# cannot consume the row-data allowance. Row-shaped indexnames use the row-data
+# work budget while retaining the metadata byte budget. Integer/Decimal bounds
+# prevent later hashing, uniqueness, and JSON conversion from allocating by magnitude.
+MAX_QUERY_RESULT_ROWS = 50_000
+MAX_QUERY_RESULT_TOTAL_ROWS = 2 * MAX_QUERY_RESULT_ROWS
+MAX_QUERY_RESULT_VALUES = 2_500_000
+MAX_QUERY_RESULT_VALUE_BYTES = 16 * 1024 * 1024
+MAX_QUERY_RESULT_METADATA_BYTES = 1024 * 1024
+MAX_QUERY_RESULT_METADATA_ITEMS = 32_768
+MAX_QUERY_RESULT_WORK = MAX_QUERY_RESULT_VALUES + MAX_QUERY_RESULT_METADATA_ITEMS
+MAX_QUERY_RESULT_STRING_BYTES = 64 * 1024
+MAX_QUERY_RESULT_KEY_BYTES = 4096
+MAX_QUERY_RESULT_INTEGER_BITS = 4096
+MAX_QUERY_RESULT_INTEGER_DIGITS = 1234
+MAX_QUERY_RESULT_DECIMAL_DIGITS = 1024
+MAX_QUERY_RESULT_DECIMAL_EXPONENT = 4096
+MAX_QUERY_RESULT_DECIMAL_STORAGE = 2048
+_BUILTIN_SCALAR_TYPES = (str, bytes, bytearray, memoryview, int, float, bool)
+_SCALAR_BASE_TYPES = (*_BUILTIN_SCALAR_TYPES, Enum)
+_SUPPORTED_COLTYPES = frozenset(GenericDataType)
+_TRUSTED_TZINFO_TYPES = (timezone, ZoneInfo)
+_DATEUTIL_TZFILE_TYPE = dateutil_tz.tzfile
+_DATEUTIL_TZOFFSET_TYPE = type(dateutil_tz.tzoffset(None, 0))
+_DATEUTIL_TZUTC_TYPE = type(dateutil_tz.UTC)
+_DATEUTIL_TZLOCAL_TYPE = type(dateutil_tz.tzlocal())
+_MAX_DATEUTIL_TRANSITIONS = 4096
+_PYTZ_FIXED_OFFSET_TYPE = type(pytz.FixedOffset(1))
+_PYTZ_UTC_TYPE = type(pytz.UTC)
+_PYTZ_NAMED_BASE_TYPES = (pytz.tzinfo.DstTzInfo, pytz.tzinfo.StaticTzInfo)
+_NUMPY_INTEGER_TYPES = frozenset(
+ type(value)
+ for value in (
+ np.int8(0),
+ np.int16(0),
+ np.int32(0),
+ np.int64(0),
+ np.uint8(0),
+ np.uint16(0),
+ np.uint32(0),
+ np.uint64(0),
+ )
+)
+_NUMPY_FLOAT_TYPES = frozenset(
+ type(value)
+ for value in (np.float16(0), np.float32(0), np.float64(0), np.longdouble(0))
+)
+_NUMPY_EXTENDED_FLOAT_TYPES = frozenset(
+ type_
+ for type_ in _NUMPY_FLOAT_TYPES
+ if np.finfo(type_).nmant > np.finfo(np.float64).nmant
+)
+_PANDAS_NAT_TYPE = type(pd.NaT)
+_PANDAS_NA_TYPE = type(pd.NA)
+_PANDAS_PERIOD_TYPE = type(pd.Period("2000-01", freq="M"))
+_PANDAS_INTERVAL_TYPE = type(pd.Interval(0, 1))
+
+
+@dataclass(frozen=True)
+class _ErrorText:
+ """Bounded error extraction outcome."""
+
+ text: str | None = None
+ malformed: str | None = None
+
+
+@dataclass
+class _ResultBudget:
+ """Aggregate work counters shared across all queries in one result."""
+
+ rows: int = 0
+ values: int = 0
+ json_bytes: int = 0
+ metadata_items: int = 0
+ metadata_bytes: int = 0
+
+
+def _truncate_utf8(value: str, max_bytes: int) -> str:
+ """Return bounded, replacement-decoded UTF-8 text.
+
+ Encoding even the non-truncated path is intentional: Python strings may
+ contain unpaired surrogates, while MCP/JSON responses must always be valid
+ UTF-8. Slicing by characters before encoding also prevents an
+ attacker-sized string from being encoded in full.
+ """
+ if max_bytes <= 0:
+ return ""
+ candidate = value[:max_bytes]
+ encoded = candidate.encode("utf-8", errors="replace")
+ if len(encoded) <= max_bytes and len(candidate) == len(value):
+ return encoded.decode("utf-8", errors="replace")
+ suffix = "... [truncated]"
+ suffix_bytes = suffix.encode()
+ if max_bytes <= len(suffix_bytes):
+ return suffix_bytes[:max_bytes].decode("ascii")
+ content_limit = max(0, max_bytes - len(suffix_bytes))
+ content = encoded[:content_limit].decode("utf-8", errors="ignore")
+ return content + suffix
+
+
+def _type_descriptor(value: Any, max_bytes: int) -> str | None:
+ """Describe an unsupported value without consulting its implementation."""
+ if max_bytes <= 0:
return None
- if isinstance(value, Mapping):
- for key in ("error", "error_message", "message", "detail"):
- if text := _query_error_text(value.get(key)):
- return text
+ value_type = type(value)
+ try:
+ type_name = type.__getattribute__(value_type, "__name__")
+ except (AttributeError, TypeError): # pragma: no cover - defensive metaclass
+ type_name = "unknown"
+ if type(type_name) is not str:
+ type_name = "unknown"
+ bounded_name = _truncate_utf8(type_name, max_bytes)
+ return _truncate_utf8(f"<{bounded_name} object>", max_bytes)
+
+
+def _type_mro(value_type: type[Any]) -> tuple[type[Any], ...]:
+ """Read a concrete type's MRO without consulting its metaclass overrides."""
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - all normal types have MRO
+ return ()
+ return mro if type(mro) is tuple else ()
+
+
+def _mro_contains(
+ value_mro: tuple[type[Any], ...], base_types: tuple[type[Any], ...]
+) -> bool:
+ """Return whether an MRO contains a base, using identity-only comparisons."""
+ return any(
+ base is expected_base for base in value_mro for expected_base in base_types
+ )
+
+
+def _safe_scalar_text(value: Any, max_bytes: int) -> str | None: # noqa: C901
+ """Render a bounded scalar without invoking attacker-controlled string code."""
+ value_type = type(value)
+ if _mro_contains(_type_mro(value_type), (Enum,)):
+ try:
+ enum_value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return _type_descriptor(value, max_bytes)
+ if not any(
+ type(enum_value) is scalar_type for scalar_type in _BUILTIN_SCALAR_TYPES
+ ):
+ return _type_descriptor(value, max_bytes)
+ return _safe_scalar_text(enum_value, max_bytes)
+ if value is None or value is False:
return None
- if isinstance(value, (list, tuple)):
- parts = [text for item in value if (text := _query_error_text(item))]
- return "; ".join(parts[:3]) or None
- text = str(value)
- return text[:2000] if text else None
+ if value_type is str:
+ return _truncate_utf8(value, max_bytes) if value else None
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ try:
+ view = memoryview(value).cast("B")
+ sample = view[: max(0, max_bytes)].tobytes()
+ text = sample.decode("utf-8", errors="replace")
+ if len(view) > len(sample):
+ text += "... [truncated]"
+ return _truncate_utf8(text, max_bytes) if text else None
+ except (TypeError, ValueError):
+ return _type_descriptor(value, max_bytes)
+ if value_type is int:
+ digits = (
+ 1 if value == 0 else int((abs(value).bit_length() - 1) * math.log10(2)) + 1
+ )
+ if digits > _MAX_INTEGER_DIGITS:
+ sign = "negative " if value < 0 else ""
+ return _truncate_utf8(
+ f"<{sign}integer with approximately {digits} decimal digits>",
+ max_bytes,
+ )
+ return _truncate_utf8(str(value), max_bytes)
+ if value_type is bool or value_type is float:
+ return _truncate_utf8(str(value), max_bytes)
+ return _type_descriptor(value, max_bytes)
+
+
+def _query_error_text(value: Any) -> _ErrorText: # noqa: C901
+ """Iteratively extract actionable text from an untrusted error payload.
+
+ Chart backends and engine adapters can return arbitrary nested error shapes.
+ Depth, visited-item, sequence-width, and output-byte limits keep validation
+ deterministic even for cycles, repeated containers, and adversarial values.
+ """
+ stack: list[tuple[Any, int]] = [(value, 0)]
+ seen: set[int] = set()
+ parts: list[str] = []
+ visited = 0
+ used_bytes = 0
+
+ while stack and len(parts) < _MAX_ERROR_PARTS:
+ item, depth = stack.pop()
+ visited += 1
+ if visited > _MAX_ERROR_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the item limit")
+ if depth > _MAX_ERROR_DEPTH:
+ return _ErrorText(malformed="error payload exceeds the depth limit")
+
+ # ChartDataCommand envelopes cross a JSON boundary. Only exact JSON
+ # containers are trusted here: ABC/isinstance checks can consult a
+ # spoofed ``__class__``, and subclass get/contains/iter/len hooks are
+ # attacker-controlled. Exact dict/list operations below are builtin and
+ # non-overridable.
+ is_mapping = type(item) is dict
+ is_sequence = type(item) is list
+ item_mro = _type_mro(type(item))
+ if not (is_mapping or is_sequence) and (
+ _mro_contains(item_mro, (dict, list, Mapping, Sequence))
+ and not _mro_contains(item_mro, _SCALAR_BASE_TYPES)
+ ):
+ return _ErrorText(
+ malformed="error payload contains an unsupported container type"
+ )
+ if is_mapping or is_sequence:
+ identity = id(item)
+ if identity in seen:
+ return _ErrorText(
+ malformed="error payload contains repeated or cyclic containers"
+ )
+ seen.add(identity)
+
+ if is_mapping:
+ children: list[Any] = []
+ for key in _ERROR_KEYS:
+ if dict.__contains__(item, key):
+ children.append(dict.__getitem__(item, key))
+ if not children:
+ if dict.__len__(item):
+ return _ErrorText(
+ malformed=(
+ "error payload object has no recognized message field"
+ )
+ )
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ if is_sequence:
+ width = list.__len__(item)
+ if width > _MAX_SEQUENCE_ITEMS:
+ return _ErrorText(malformed="error payload exceeds the width limit")
+ children = [list.__getitem__(item, index) for index in range(width)]
+ stack.extend((child, depth + 1) for child in reversed(children))
+ continue
+
+ remaining = _MAX_ERROR_BYTES - used_bytes - (2 if parts else 0)
+ text = _safe_scalar_text(item, remaining)
+ if text:
+ parts.append(text)
+ used_bytes += len(text.encode("utf-8", errors="replace")) + (
+ 2 if len(parts) > 1 else 0
+ )
+
+ return _ErrorText(text="; ".join(parts) or None)
-def _failure_for_query_payload(
- payload: Mapping[str, Any], label: str
+def _failure_for_query_payload( # noqa: C901
+ payload: dict[str, Any], label: str
) -> ChartError | None:
"""Extract one failure from a top-level or per-query payload."""
+ malformed: str | None = None
for key in ("error", "errors", "error_message"):
- if message := _query_error_text(payload.get(key)):
+ extracted = _query_error_text(dict.get(payload, key))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ continue
+ if message := extracted.text:
return ChartError(
error=f"{label} failed: {message}", error_type="QueryError"
)
- raw_status = payload.get("status")
- status = str(getattr(raw_status, "value", raw_status) or "")
+ raw_status = dict.get(payload, "status")
+ status = _safe_scalar_text(raw_status, 200) or ""
normalized_status = status.strip().casefold().replace("-", "_").replace(" ", "_")
if normalized_status in FAILED_QUERY_STATUSES:
- message = (
- _query_error_text(payload.get("message"))
- or _query_error_text(payload.get("error_message"))
- or normalized_status
- )
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ fallback = _query_error_text(dict.get(payload, "error_message"))
+ if fallback.malformed:
+ malformed = malformed or fallback.malformed
+ if malformed and not (extracted.text or fallback.text):
+ return _malformed_result(malformed)
+ message = extracted.text or fallback.text or normalized_status
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
- if payload.get("success") is False:
- message = _query_error_text(payload.get("message")) or "request failed"
+ if dict.get(payload, "success") is False:
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if malformed and not extracted.text:
+ return _malformed_result(malformed)
+ message = extracted.text or "request failed"
return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
if (
raw_status is None
- and "data" not in payload
- and "queries" not in payload
- and (message := _query_error_text(payload.get("message")))
+ and "data" not in dict.keys(payload)
+ and "queries" not in dict.keys(payload)
):
- return ChartError(error=f"{label} failed: {message}", error_type="QueryError")
+ extracted = _query_error_text(dict.get(payload, "message"))
+ if extracted.malformed:
+ malformed = malformed or extracted.malformed
+ if extracted.text:
+ return ChartError(
+ error=f"{label} failed: {extracted.text}", error_type="QueryError"
+ )
+ if malformed:
+ return _malformed_result(malformed)
return None
-def query_result_failure(result: Any) -> ChartError | None:
- """Return a structured failure embedded in a ChartDataCommand payload.
+def _malformed_result(message: str) -> ChartError:
+ """Build a stable error for an invalid ChartDataCommand envelope."""
+ return ChartError(
+ error=f"Malformed chart query result: {message}",
+ error_type="MalformedQueryResult",
+ )
- ChartDataCommand can return an HTTP-successful envelope whose top level or
- any query reports a failure. Every query is inspected before callers accept
- data from the result. Successful statuses may carry informational messages,
- so ``message`` alone is not treated as an error.
+
+def bounded_result_row_count(value: Any) -> int | None:
+ """Return one exact bounded row count, rejecting coercive lookalikes."""
+ if value is None:
+ return None
+ if type(value) is int:
+ count = value
+ elif type(value) is float and math.isfinite(value) and value.is_integer():
+ count = int(value)
+ else:
+ raise ValueError("must be a finite non-negative integral number")
+ if count < 0:
+ raise ValueError("must be non-negative")
+ if count > _MAX_RESULT_ROW_COUNT:
+ raise ValueError("exceeds the supported bound")
+ return count
+
+
+def _bounded_utf8_length(value: str, max_bytes: int) -> int | None:
+ """Return an exact UTF-8 size without encoding attacker-sized text."""
+ if str.__len__(value) > max_bytes:
+ return None
+ try:
+ encoded = str.encode(value, "utf-8", errors="strict")
+ except UnicodeEncodeError:
+ return None
+ size = bytes.__len__(encoded)
+ return size if size <= max_bytes else None
+
+
+def _json_string_size(value: str, max_bytes: int) -> int | None:
+ """Return the exact UTF-8 size of a JSON string without serializing it."""
+ raw_size = _bounded_utf8_length(value, max_bytes)
+ if raw_size is None:
+ return None
+ escaped_size = raw_size + 2 # surrounding quotes
+ for character in value:
+ codepoint = ord(character)
+ if character in {'"', "\\"} or character in {"\b", "\t", "\n", "\f", "\r"}:
+ escaped_size += 1
+ elif codepoint < 0x20:
+ # Other JSON control characters use a six-byte ``\\u00xx`` escape.
+ escaped_size += 5
+ return escaped_size
+
+
+def _integer_json_size(value: int) -> int:
+ """Return an exact integer JSON size without creating its decimal string."""
+ magnitude = -value if value < 0 else value
+ if magnitude == 0:
+ digits = 1
+ else:
+ bits = int.bit_length(magnitude)
+ # This fixed-point log10(2) estimate is at most one digit low. Refine it
+ # with one bounded integer comparison rather than rendering the value.
+ digits = ((bits - 1) * 30103) // 100000 + 1
+ if magnitude >= 10**digits:
+ digits += 1
+ return digits + (value < 0)
+
+
+def _container_json_syntax_size(item_count: int, *, mapping: bool) -> int:
+ """Return braces/brackets, separators, and mapping-colon byte cost."""
+ if item_count == 0:
+ return 2
+ return 2 + item_count - 1 + (item_count if mapping else 0)
+
+
+def _trusted_timedelta_text(value: timedelta) -> str:
+ """Render an exact timedelta with Pydantic's stable ISO-8601 spelling."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ years, days = divmod(days, 365)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+
+ date_parts = [f"{years}Y" if years else "", f"{days}D" if days else ""]
+ time_parts = [f"{hours}H" if hours else "", f"{minutes}M" if minutes else ""]
+ if microseconds:
+ fraction = f"{microseconds:06d}".rstrip("0")
+ time_parts.append(f"{seconds}.{fraction}S")
+ elif seconds:
+ time_parts.append(f"{seconds}S")
+
+ date_text = "".join(date_parts)
+ time_text = "".join(time_parts)
+ if not date_text and not time_text:
+ time_text = "0S"
+ return f"{sign}P{date_text}{'T' if time_text else ''}{time_text}"
+
+
+def _chart_data_builtin_timedelta_text(value: timedelta) -> str:
+ """Reproduce ``format_timedelta`` without comparison or string hooks."""
+ total_microseconds = (
+ value.days * 86_400 + value.seconds
+ ) * 1_000_000 + value.microseconds
+ sign = "-" if total_microseconds < 0 else ""
+ remaining = abs(total_microseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000)
+ seconds, microseconds = divmod(remaining, 1_000_000)
+ day_text = f"{days} {'day' if days == 1 else 'days'}, " if days else ""
+ fraction = f".{microseconds:06d}" if microseconds else ""
+ return f"{sign}{day_text}{hours}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _chart_data_pandas_timedelta_text(value: pd.Timedelta) -> str:
+ """Reproduce Chart Data ``format_timedelta`` output from exact fields."""
+ total_nanoseconds = (
+ (
+ object.__getattribute__(value, "days") * 86_400
+ + object.__getattribute__(value, "seconds")
+ )
+ * 1_000_000
+ + object.__getattribute__(value, "microseconds")
+ ) * 1_000 + object.__getattribute__(value, "nanoseconds")
+ sign = "-" if total_nanoseconds < 0 else ""
+ remaining = abs(total_nanoseconds)
+ days, remaining = divmod(remaining, 86_400 * 1_000_000_000)
+ hours, remaining = divmod(remaining, 3_600 * 1_000_000_000)
+ minutes, remaining = divmod(remaining, 60 * 1_000_000_000)
+ seconds, nanoseconds = divmod(remaining, 1_000_000_000)
+ if nanoseconds % 1_000:
+ fraction = f".{nanoseconds:09d}"
+ elif nanoseconds:
+ fraction = f".{nanoseconds // 1_000:06d}"
+ else:
+ fraction = ""
+ return f"{sign}{days} days {hours:02d}:{minutes:02d}:{seconds:02d}{fraction}"
+
+
+def _normalized_scalar_json_size( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> int:
+ """Return a conservative encoded size for one normalized exact scalar."""
+ value_type = type(value)
+ if value is None:
+ return 4
+ if value_type is bool:
+ return 4 if value else 5
+ if value_type is str:
+ size = _json_string_size(value, max_string_bytes)
+ assert size is not None # scalar normalization already bounded the string
+ return size
+ if value_type is int:
+ return _integer_json_size(value)
+ if value_type is float:
+ if not math.isfinite(value):
+ # Raw Gauge exports retain these markers; JSON responses use null.
+ return 4
+ # Exact builtin repr is hook-free, bounded to a shortest-round-trip
+ # spelling, and avoids pessimistically charging 24 bytes for values
+ # such as 0.0 across ordinary large numeric datasets.
+ return len(float.__repr__(value))
+ if value_type is Decimal:
+ # Decimal storage, coefficient digits, and exponent are bounded before
+ # this point. Its canonical spelling is therefore itself bounded, and
+ # Pydantic serializes Decimal values as JSON strings.
+ text = Decimal.__str__(value)
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+ if value_type is datetime:
+ return 40
+ if value_type is date:
+ text = date.isoformat(value)
+ elif value_type is time:
+ return 32
+ elif value_type is timedelta:
+ text = _trusted_timedelta_text(value)
+ elif value_type is UUID:
+ text = UUID.__str__(value)
+ else:
+ raise AssertionError(f"unaccounted normalized scalar: {value_type!r}")
+ size = _json_string_size(text, MAX_QUERY_RESULT_STRING_BYTES)
+ assert size is not None
+ return size
+
+
+def _pydantic_scalar_json_size(value: Any) -> int:
+ """Return the exact Pydantic wire size for a normalized scalar.
+
+ Source-result accounting deliberately retains its existing conservative
+ scalar rules. Final response projections, however, must match
+ pydantic-core's JSON number spelling: for example, it emits ``0.00001`` for
+ ``1e-5`` and ``1e-6`` for ``1e-6`` rather than Python's repr spellings.
"""
- if not isinstance(result, Mapping):
+ if type(value) is float:
+ return len(to_json(value))
+ return _normalized_scalar_json_size(value)
+
+
+def _charge_json_bytes(
+ budget: _ResultBudget, size: int, *, metadata: bool = False
+) -> str | None:
+ """Charge aggregate response bytes and the independent metadata allowance."""
+ budget.json_bytes += size
+ if budget.json_bytes > MAX_QUERY_RESULT_VALUE_BYTES:
+ return "exceeds the total JSON-encoded byte limit"
+ if metadata:
+ budget.metadata_bytes += size
+ if budget.metadata_bytes > MAX_QUERY_RESULT_METADATA_BYTES:
+ return "metadata exceeds the total JSON-encoded byte limit"
+ return None
+
+
+def _integer_failure(value: int) -> str | None:
+ """Validate exact integer magnitude before decimal rendering or hashing."""
+ bits = int.bit_length(value)
+ if bits > MAX_QUERY_RESULT_INTEGER_BITS:
+ return "contains an integer exceeding the bit-length limit"
+ digits = 1 if bits == 0 else ((bits - 1) * 30103) // 100000 + 1
+ if digits > MAX_QUERY_RESULT_INTEGER_DIGITS:
+ return "contains an integer exceeding the digit limit"
+ return None
+
+
+def _decimal_failure(value: Decimal) -> str | None:
+ """Validate exact Decimal storage, finiteness, digits, and exponent."""
+ if Decimal.__sizeof__(value) > MAX_QUERY_RESULT_DECIMAL_STORAGE:
+ return "contains a Decimal exceeding the storage limit"
+ if not Decimal.is_finite(value):
+ return "contains a non-finite Decimal"
+ parts = Decimal.as_tuple(value)
+ if tuple.__len__(parts.digits) > MAX_QUERY_RESULT_DECIMAL_DIGITS:
+ return "contains a Decimal exceeding the digit limit"
+ exponent = parts.exponent
+ if type(exponent) is not int or abs(exponent) > MAX_QUERY_RESULT_DECIMAL_EXPONENT:
+ return "contains a Decimal exceeding the exponent limit"
+ return None
+
+
+def _exact_object_namespace(value: Any) -> dict[str, Any] | None:
+ """Read an object's concrete storage without descriptor dispatch."""
+ try:
+ namespace = object.__getattribute__(value, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ return namespace if type(namespace) is dict else None
+
+
+def _dateutil_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read a dateutil tzfile's IANA name from exact internal storage."""
+ tzinfo_type = type(tzinfo)
+ if tzinfo_type not in {_DATEUTIL_TZFILE_TYPE, dateutil_zoneinfo_tzfile}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ filename = dict.get(namespace, "_filename")
+ if type(filename) is not str or _bounded_utf8_length(filename, 4096) is None:
+ return None
+ if tzinfo_type is dateutil_zoneinfo_tzfile:
+ name = filename
+ else:
+ marker = "/zoneinfo/"
+ marker_offset = str.find(filename, marker)
+ if marker_offset >= 0:
+ name = str.__getitem__(filename, slice(marker_offset + len(marker), None))
+ elif not str.startswith(filename, "/") and str.find(filename, "\\") < 0:
+ name = filename
+ else:
+ return None
+ parts = str.split(name, "/")
+ if not parts or any(part in {"", ".", ".."} for part in parts):
+ return None
+ return name if _bounded_utf8_length(name, 256) is not None else None
+
+
+def _dateutil_ttinfo_without_hooks(
+ value: Any,
+) -> tuple[int, timedelta] | None:
+ """Read one exact dateutil transition record without user-hook dispatch."""
+ if type(value) is not dateutil_ttinfo:
+ return None
+ try:
+ offset = object.__getattribute__(value, "offset")
+ delta = object.__getattribute__(value, "delta")
+ except (AttributeError, TypeError):
+ return None
+ if type(offset) is not int or type(delta) is not timedelta:
+ return None
+ try:
+ if delta != timedelta(seconds=offset):
+ return None
+ except OverflowError:
+ return None
+ return offset, delta
+
+
+def _dateutil_named_offset_without_hooks( # noqa: C901
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Recover the offset selected by an exact dateutil named timezone.
+
+ A dateutil tzfile's finite transition table is its wire-semantic source of
+ truth. Reinterpreting its wall time through a system ``ZoneInfo`` database
+ changes negative-DST folds, nonexistent times, and dates after the final
+ transition. This mirrors dateutil's transition selection using only exact
+ builtin containers and its exact trusted transition-record type.
+ """
+ if _dateutil_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ transitions = dict.get(namespace, "_trans_list")
+ transition_info = dict.get(namespace, "_trans_idx")
+ standard_info = dict.get(namespace, "_ttinfo_std")
+ before_info = dict.get(namespace, "_ttinfo_before")
+ transition_count = tuple.__len__(transitions) if type(transitions) is tuple else 0
+ if (
+ type(transitions) is not tuple
+ or type(transition_info) is not tuple
+ or tuple.__len__(transitions) != tuple.__len__(transition_info)
+ or transition_count > _MAX_DATEUTIL_TRANSITIONS
+ or _dateutil_ttinfo_without_hooks(standard_info) is None
+ or (
+ transition_count > 0 and _dateutil_ttinfo_without_hooks(before_info) is None
+ )
+ ):
+ return None
+
+ previous: int | None = None
+ for transition in transitions:
+ if (
+ type(transition) is not int
+ or int.bit_length(transition) > 63
+ or (previous is not None and transition < previous)
+ ):
+ return None
+ previous = transition
+ if any(_dateutil_ttinfo_without_hooks(info) is None for info in transition_info):
+ return None
+
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ try:
+ timestamp = (naive - EPOCH).total_seconds()
+ except (OverflowError, TypeError, ValueError):
+ return None
+
+ index: int | None = (
+ bisect_right(transitions, timestamp) - 1 if transition_count else None
+ )
+
+ def info_at(selected: int | None) -> Any:
+ if selected is None or selected + 1 >= transition_count:
+ return standard_info
+ if selected < 0:
+ return before_info
+ return tuple.__getitem__(transition_info, selected)
+
+ if index is not None and index != 0:
+ current = _dateutil_ttinfo_without_hooks(info_at(index))
+ prior = _dateutil_ttinfo_without_hooks(info_at(index - 1))
+ if current is None or prior is None:
+ return None
+ offset_delta = prior[0] - current[0]
+ transition = tuple.__getitem__(transitions, index)
+ is_ambiguous = timestamp < transition + offset_delta
+ index -= int(not value.fold and is_ambiguous)
+
+ selected = _dateutil_ttinfo_without_hooks(info_at(index))
+ if selected is None:
+ return None
+ try:
+ return timezone(selected[1])
+ except ValueError:
+ return None
+
+
+def _pytz_timezone_name_without_hooks(tzinfo: Any) -> str | None:
+ """Read and verify one generated pytz named-zone implementation."""
+ value_type = type(tzinfo)
+ if not _mro_contains(_type_mro(value_type), _PYTZ_NAMED_BASE_TYPES):
+ return None
+ try:
+ namespace = type.__getattribute__(value_type, "__dict__")
+ except (AttributeError, TypeError):
+ return None
+ if type(namespace) is not MappingProxyType:
+ return None
+ zone = namespace.get("zone")
+ if type(zone) is not str or _bounded_utf8_length(zone, 256) is None:
+ return None
+ try:
+ canonical = pytz.timezone(zone)
+ except (KeyError, ValueError):
+ return None
+ # A user subclass can inherit pytz's base and spoof ``zone``. Only the
+ # concrete class generated and cached by pytz for that name is trusted.
+ return zone if type(canonical) is value_type else None
+
+
+def _fixed_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Reconstruct trusted dateutil/pytz fixed offsets from exact storage."""
+ if type(tzinfo) not in {_DATEUTIL_TZOFFSET_TYPE, _PYTZ_FIXED_OFFSET_TYPE}:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_offset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _pytz_named_offset_without_hooks(tzinfo: Any) -> timezone | None:
+ """Return a localized pytz instance's stored offset without its hooks."""
+ if _pytz_timezone_name_without_hooks(tzinfo) is None:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ offset = dict.get(namespace, "_utcoffset")
+ if type(offset) is not timedelta:
+ return None
+ try:
+ return timezone(offset)
+ except ValueError:
+ return None
+
+
+def _dateutil_local_offset_without_hooks(
+ value: datetime, tzinfo: Any
+) -> timezone | None:
+ """Select an exact dateutil-local offset using builtin system time data."""
+ if type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE:
+ return None
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None:
+ return None
+ standard_offset = dict.get(namespace, "_std_offset")
+ daylight_offset = dict.get(namespace, "_dst_offset")
+ has_daylight = dict.get(namespace, "_hasdst")
+ if (
+ type(standard_offset) is not timedelta
+ or type(daylight_offset) is not timedelta
+ or type(has_daylight) is not bool
+ ):
+ return None
+ selected_offset = standard_offset
+ if has_daylight:
+ epoch = datetime(1970, 1, 1)
+ naive = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ )
+ timestamp = (naive - epoch).total_seconds()
+ try:
+ is_daylight = bool(
+ system_time.localtime(timestamp + system_time.timezone).tm_isdst
+ )
+ daylight_saved = daylight_offset - standard_offset
+ previous_is_daylight = bool(
+ system_time.localtime(
+ timestamp
+ - timedelta.total_seconds(daylight_saved)
+ + system_time.timezone
+ ).tm_isdst
+ )
+ except (OverflowError, OSError, ValueError):
+ return None
+ is_ambiguous = not is_daylight and is_daylight != previous_is_daylight
+ if is_ambiguous:
+ is_daylight = not bool(value.fold)
+ selected_offset = daylight_offset if is_daylight else standard_offset
+ try:
+ return timezone(selected_offset)
+ except ValueError:
+ return None
+
+
+def _canonical_timezone(tzinfo: Any) -> timezone | ZoneInfo | None:
+ """Return an exact trusted timezone without invoking the source's methods."""
+ if any(type(tzinfo) is type_ for type_ in _TRUSTED_TZINFO_TYPES):
+ return tzinfo
+ if type(tzinfo) in {_DATEUTIL_TZUTC_TYPE, _PYTZ_UTC_TYPE}:
+ return timezone.utc
+ if fixed_offset := _fixed_offset_without_hooks(tzinfo):
+ return fixed_offset
+ zone_name = _dateutil_timezone_name_without_hooks(
+ tzinfo
+ ) or _pytz_timezone_name_without_hooks(tzinfo)
+ if zone_name:
+ try:
+ return ZoneInfo(zone_name)
+ except (KeyError, ValueError, ZoneInfoNotFoundError):
+ return None
+ return None
+
+
+def _timestamp_offset_without_hooks(value: pd.Timestamp) -> timezone | None:
+ """Recover a timestamp's stored wall-clock offset without timezone hooks."""
+ unit_multipliers = {"s": 1_000_000_000, "ms": 1_000_000, "us": 1_000, "ns": 1}
+ multiplier = unit_multipliers.get(value.unit)
+ if multiplier is None:
+ return None
+ try:
+ instant_ns = int(value.asm8.view("i8")) * multiplier
+ epoch_ordinal = date.toordinal(date(1970, 1, 1))
+ wall_ns = (
+ (
+ (datetime.toordinal(value) - epoch_ordinal) * 86_400
+ + value.hour * 3600
+ + value.minute * 60
+ + value.second
+ )
+ * 1_000_000_000
+ + value.microsecond * 1000
+ + value.nanosecond
+ )
+ offset_ns = wall_ns - instant_ns
+ if offset_ns % 1000:
+ return None
+ return timezone(timedelta(microseconds=offset_ns // 1000))
+ except (OverflowError, TypeError, ValueError):
return None
+
+def _trusted_datetime_value(
+ value: datetime,
+) -> tuple[datetime | None, str | None]:
+ """Return an exact datetime rebuilt with only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz: timezone | ZoneInfo | None
+ if _dateutil_timezone_name_without_hooks(tzinfo) is not None:
+ # A recognized dateutil tzfile must use its own finite transition
+ # table. Falling through to ZoneInfo would silently reinterpret a
+ # source-selected gap/fold or post-table wall time.
+ canonical_tz = _dateutil_named_offset_without_hooks(value, tzinfo)
+ else:
+ canonical_tz = (
+ _pytz_named_offset_without_hooks(tzinfo)
+ or _dateutil_local_offset_without_hooks(value, tzinfo)
+ or _canonical_timezone(tzinfo)
+ )
+ if canonical_tz is None:
+ return None, "contains a datetime with an unsupported timezone"
+ canonical_value = datetime(
+ value.year,
+ value.month,
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ # Exercise builtin validation without dispatching through a source
+ # timezone after the reconstruction above.
+ datetime.isoformat(canonical_value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ return canonical_value, None
+
+
+def _trusted_datetime_text(value: datetime) -> tuple[str | None, str | None]:
+ """Serialize an exact Python datetime through only trusted timezone types."""
+ canonical_value, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid datetime"
+ return datetime.isoformat(canonical_value), None
+
+
+def _trusted_time_text(value: time) -> tuple[str | None, str | None]:
+ """Serialize an exact Python time through only trusted timezone types."""
+ tzinfo = value.tzinfo
+ canonical_value = value
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ canonical_tz = _canonical_timezone(tzinfo)
+ if canonical_tz is None and type(tzinfo) is _DATEUTIL_TZLOCAL_TYPE:
+ namespace = _exact_object_namespace(tzinfo)
+ if namespace is None or type(dict.get(namespace, "_hasdst")) is not bool:
+ return None, "contains a time with an unsupported timezone"
+ if dict.get(namespace, "_hasdst"):
+ canonical_tz = None
+ else:
+ standard_offset = dict.get(namespace, "_std_offset")
+ if type(standard_offset) is not timedelta:
+ return None, "contains a time with an unsupported timezone"
+ try:
+ canonical_tz = timezone(standard_offset)
+ except ValueError:
+ return None, "contains a time with an unsupported timezone"
+ elif canonical_tz is None:
+ return None, "contains a time with an unsupported timezone"
+ canonical_value = time(
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond,
+ tzinfo=canonical_tz,
+ fold=value.fold,
+ )
+ try:
+ return time.isoformat(canonical_value), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid time"
+
+
+def _trusted_timestamp_value(
+ value: pd.Timestamp,
+) -> tuple[pd.Timestamp | None, str | None]:
+ """Return a timestamp rebuilt with only trusted timezone implementations."""
+ tzinfo = value.tzinfo
+ try:
+ if tzinfo is not None and not any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ if (
+ _canonical_timezone(tzinfo) is None
+ and type(tzinfo) is not _DATEUTIL_TZLOCAL_TYPE
+ ):
+ return (
+ None,
+ "contains a pandas timestamp with an unsupported timezone",
+ )
+ if (canonical_tz := _timestamp_offset_without_hooks(value)) is None:
+ return None, "contains an invalid pandas timestamp"
+ # Rebuild from the stored instant and resolution. No method on the
+ # original pytz/dateutil object is called, and the recovered fixed
+ # offset preserves the timestamp's selected fold.
+ raw_value = value.asm8.view("i8")
+ value = pd.Timestamp(raw_value, unit=value.unit, tz="UTC").tz_convert(
+ canonical_tz
+ )
+ # Validate the retained resolution and selected UTC offset.
+ pd.Timestamp.isoformat(value)
+ except (KeyError, OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ return value, None
+
+
+def _trusted_timestamp_text(value: pd.Timestamp) -> tuple[str | None, str | None]:
+ """Convert an exact pandas timestamp to its canonical JSON representation."""
+ canonical_value, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_value is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ # ISO output preserves nanoseconds and the UTC offset selected by fold.
+ text = pd.Timestamp.isoformat(canonical_value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timestamp"
+ return text, None
+
+
+def _normalize_trusted_scalar( # noqa: C901
+ value: Any, *, max_string_bytes: int = MAX_QUERY_RESULT_STRING_BYTES
+) -> tuple[Any, str | None]:
+ """Normalize one exact trusted pandas/NumPy scalar or validate a builtin.
+
+ Type identity is checked before every conversion. This deliberately does not
+ accept subclasses or generic ``np.generic``/pandas extension objects, whose
+ conversion hooks are outside the trusted ChartData materialization contract.
+ """
+ value_type = type(value)
+ enum_seen: set[int] = set()
+ while _mro_contains(_type_mro(value_type), (Enum,)):
+ identity = id(value)
+ if identity in enum_seen or len(enum_seen) >= _MAX_ROW_CONTAINER_DEPTH:
+ return None, "contains a recursive enum"
+ enum_seen.add(identity)
+ try:
+ value = object.__getattribute__(value, "_value_")
+ except Exception:
+ return None, "contains an unsupported enum"
+ value_type = type(value)
+
+ if value is None or value_type is bool:
+ return value, None
+ if value_type is str:
+ size = _bounded_utf8_length(value, max_string_bytes)
+ return (
+ (value, None)
+ if size is not None
+ else (
+ None,
+ "contains an invalid or oversized string",
+ )
+ )
+ if value_type is int:
+ return value, _integer_failure(value)
+ if value_type is float:
+ if math.isnan(value):
+ return None, None
+ if math.isinf(value):
+ return None, "contains a non-finite number"
+ return value, None
+ if value_type is Decimal:
+ return value, _decimal_failure(value)
+
+ if value_type is datetime:
+ return _trusted_datetime_text(value)
+ if value_type is time:
+ return _trusted_time_text(value)
+ if value_type is date:
+ return date.isoformat(value), None
+ if value_type is timedelta:
+ return _trusted_timedelta_text(value), None
+ if value_type is UUID:
+ return UUID.__str__(value), None
+
+ if value_type is _PANDAS_NAT_TYPE or value_type is _PANDAS_NA_TYPE:
+ return None, None
+ if value_type is pd.Timestamp:
+ return _trusted_timestamp_text(value)
+ if value_type is pd.Timedelta:
+ if pd.isna(value):
+ return None, None
+ text = pd.Timedelta.isoformat(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas timedelta"
+ return text, None
+ if value_type is _PANDAS_PERIOD_TYPE or value_type is _PANDAS_INTERVAL_TYPE:
+ # The concrete extension scalar implementations are trusted, unlike an
+ # arbitrary subclass's ``__str__`` implementation.
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized pandas scalar"
+ return text, None
+
+ if any(value_type is type_ for type_ in _NUMPY_INTEGER_TYPES):
+ normalized_integer = int(value)
+ return normalized_integer, _integer_failure(normalized_integer)
+ if any(value_type is type_ for type_ in _NUMPY_EXTENDED_FLOAT_TYPES):
+ if np.isnan(value):
+ return None, None
+ if not np.isfinite(value):
+ return None, "contains a non-finite NumPy number"
+ # JSON has no extended floating-point type. Preserve the trusted scalar
+ # as a round-trippable decimal string instead of narrowing to binary64.
+ return np.format_float_scientific(value, unique=True, trim="-"), None
+ if any(value_type is type_ for type_ in _NUMPY_FLOAT_TYPES):
+ normalized_float = float(value)
+ if math.isnan(normalized_float):
+ return None, None
+ if math.isinf(normalized_float):
+ return None, "contains a non-finite NumPy number"
+ return normalized_float, None
+ if value_type is np.bool_:
+ return bool(value), None
+ if value_type is np.str_:
+ text = str(value)
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an invalid or oversized NumPy string"
+ return text, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ timestamp = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ return _trusted_timestamp_text(timestamp)
+ if value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ delta = pd.Timedelta(value)
+ text = pd.Timedelta.isoformat(delta)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy timedelta"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized NumPy timedelta"
+ return text, None
+
+ if value_type is bytes or value_type is bytearray or value_type is memoryview:
+ # Exact binary column values follow the documented serialization
+ # contract: UTF-8 text, otherwise a ``base64:``-prefixed string.
+ try:
+ raw_size = value.nbytes if value_type is memoryview else len(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if raw_size > max_string_bytes:
+ return None, "contains an oversized binary value"
+ try:
+ text = decode_binary(value)
+ except (TypeError, ValueError):
+ return None, "contains an unreadable binary value"
+ if _bounded_utf8_length(text, max_string_bytes) is None:
+ return None, "contains an oversized binary value"
+ return text, None
+
+ return None, "contains an unsupported or subclassed value"
+
+
+def _is_chart_data_temporal_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data temporal wire semantics."""
+ return type(value) in {
+ date,
+ datetime,
+ pd.Timestamp,
+ np.datetime64,
+ _PANDAS_NAT_TYPE,
+ }
+
+
+def _is_chart_data_duration_scalar(value: Any) -> bool:
+ """Return whether an exact scalar has Chart Data duration semantics."""
+ return type(value) in {timedelta, pd.Timedelta, np.timedelta64}
+
+
+def _chart_data_duration_text(value: Any) -> tuple[str | None, str | None]:
+ """Project an exact duration through Chart Data's public JSON spelling.
+
+ Builtin and pandas durations are ``timedelta`` instances consumed by
+ ``format_timedelta``. Exact NumPy durations model the real DataFrame
+ producer boundary: unambiguous units are promoted to a pandas Timedelta,
+ NaT becomes JSON null, and ambiguous or overflowing units fail closed.
+ """
+ value_type = type(value)
+ if value_type is timedelta:
+ text = _chart_data_builtin_timedelta_text(value)
+ elif value_type is pd.Timedelta:
+ text = _chart_data_pandas_timedelta_text(value)
+ elif value_type is np.timedelta64:
+ if np.isnat(value):
+ return None, None
+ try:
+ pandas_value = pd.Timedelta(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy duration"
+ text = _chart_data_pandas_timedelta_text(pandas_value)
+ else:
+ return None, "contains an unsupported duration value"
+ if _bounded_utf8_length(text, MAX_QUERY_RESULT_STRING_BYTES) is None:
+ return None, "contains an oversized duration"
+ return text, None
+
+
+def _chart_data_temporal_number( # noqa: C901
+ value: Any,
+) -> tuple[float | None, str | None]:
+ """Project an exact date/datetime through Chart Data's epoch-ms wire form.
+
+ The public Chart Data API uses ``json_int_dttm_ser`` before the browser
+ parses the payload. Trusted canonicalizers first validate or replace
+ timezone implementations without arbitrary hooks, then the production
+ ``datetime_to_epoch`` helper preserves its exact float behavior.
+ """
+ value_type = type(value)
+ if value_type is _PANDAS_NAT_TYPE:
+ # json_int_dttm_ser produces NaN and the Chart Data response's
+ # ``ignore_nan=True`` projects it to JSON null.
+ return None, None
+ if value_type is np.datetime64:
+ if np.isnat(value):
+ return None, None
+ try:
+ value = pd.Timestamp(value)
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid NumPy datetime"
+ value_type = type(value)
+ if value_type is pd.Timestamp:
+ canonical_timestamp, reason = _trusted_timestamp_value(value)
+ if reason is not None or canonical_timestamp is None:
+ return None, reason or "contains an invalid pandas timestamp"
+ try:
+ return datetime_to_epoch(canonical_timestamp), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid pandas timestamp"
+ if value_type is datetime:
+ canonical_datetime, reason = _trusted_datetime_value(value)
+ if reason is not None or canonical_datetime is None:
+ return None, reason or "contains an invalid datetime"
+ try:
+ return datetime_to_epoch(canonical_datetime), None
+ except (OverflowError, TypeError, ValueError):
+ return None, "contains an invalid datetime"
+ if value_type is date:
+ return (value - EPOCH.date()).total_seconds() * 1000, None
+ return None, "contains an unsupported temporal value"
+
+
+def _add_result_value_to_budget(value: Any, budget: _ResultBudget) -> str | None:
+ """Account for one normalized row value and its JSON scalar size."""
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "contains too many total values"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "exceeds the total work limit"
+ if type(value) is not list and type(value) is not dict:
+ if reason := _charge_json_bytes(budget, _normalized_scalar_json_size(value)):
+ return reason
+ return None
+
+
+def _metadata_failure_for_value( # noqa: C901
+ value: Any, budget: _ResultBudget, *, row_shaped: bool = False
+) -> str | None:
+ """Bound metadata, charging row-shaped indexes to the row-data work budget."""
+ stack: list[tuple[Any, int, bool]] = [(value, 0, False)]
+ active_containers: set[int] = set()
+ while stack:
+ item, depth, leaving = stack.pop()
+ if leaving:
+ active_containers.remove(id(item))
+ continue
+ if row_shaped:
+ budget.values += 1
+ if budget.values > MAX_QUERY_RESULT_VALUES:
+ return "metadata contains too many total values"
+ else:
+ budget.metadata_items += 1
+ if budget.metadata_items > MAX_QUERY_RESULT_METADATA_ITEMS:
+ return "metadata exceeds the item limit"
+ if budget.values + budget.metadata_items > MAX_QUERY_RESULT_WORK:
+ return "metadata exceeds the total work limit"
+ if depth > _MAX_ROW_CONTAINER_DEPTH:
+ return "metadata exceeds the nesting depth limit"
+
+ if type(item) is dict:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ item_count = dict.__len__(item)
+ if item_count > _MAX_ROW_CONTAINER_ITEMS:
+ return "metadata contains an oversized object"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(item_count, mapping=True),
+ metadata=True,
+ ):
+ return reason
+ for key, child in dict.items(item):
+ if type(key) is not str:
+ return "metadata contains a non-string object key"
+ key_size = _json_string_size(key, MAX_QUERY_RESULT_KEY_BYTES)
+ if key_size is None:
+ return "metadata contains an invalid or oversized object key"
+ if reason := _charge_json_bytes(budget, key_size, metadata=True):
+ return reason
+ stack.append((child, depth + 1, False))
+ continue
+
+ if type(item) is list or type(item) is tuple:
+ identity = id(item)
+ if identity in active_containers:
+ return "metadata contains cyclic containers"
+ active_containers.add(identity)
+ stack.append((item, depth, True))
+ width = len(item)
+ # Chart Data emits one indexname per row. Only that outer array
+ # receives the row limit; containers within an index stay bounded.
+ max_items = (
+ MAX_QUERY_RESULT_ROWS
+ if row_shaped and depth == 0
+ else _MAX_ROW_CONTAINER_ITEMS
+ )
+ if width > max_items:
+ return "metadata contains an oversized array"
+ if reason := _charge_json_bytes(
+ budget,
+ _container_json_syntax_size(width, mapping=False),
+ metadata=True,
+ ):
+ return reason
+ stack.extend((item[index], depth + 1, False) for index in range(width))
+ continue
+
+ normalized, reason = _normalize_trusted_scalar(
+ item, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ )
+ if reason is not None:
+ return f"metadata {reason}"
+ if reason := _charge_json_bytes(
+ budget,
+ _normalized_scalar_json_size(
+ normalized, max_string_bytes=MAX_QUERY_RESULT_METADATA_BYTES
+ ),
+ metadata=True,
+ ):
+ return reason
+ return None
+
+
+def _metadata_failure( # noqa: C901
+ payload: dict[str, Any], label: str
+) -> ChartError | None:
+ """Validate bounded cache and row-count metadata before any consumer."""
+ for key in ("rowcount", "total_rows"):
+ if key in payload and dict.__getitem__(payload, key) is not None:
+ try:
+ bounded_result_row_count(dict.__getitem__(payload, key))
+ except ValueError as ex:
+ return _malformed_result(f"{label} {key} {ex}")
+
+ if "is_cached" in payload:
+ is_cached = dict.__getitem__(payload, "is_cached")
+ if is_cached is not None and type(is_cached) is not bool:
+ return _malformed_result(f"{label} is_cached must be an exact boolean")
+
+ if "cache_key" in payload:
+ cache_key = dict.__getitem__(payload, "cache_key")
+ if cache_key is not None and (
+ type(cache_key) is not str
+ or _bounded_utf8_length(cache_key, _MAX_CACHE_STRING_BYTES) is None
+ ):
+ return _malformed_result(
+ f"{label} cache_key must be a bounded exact string"
+ )
+
+ # ChartData's production schema emits ``cached_dttm``. ``cache_dttm`` was
+ # used by earlier MCP payloads and remains a bounded compatibility alias.
+ # Validate both before cache utilities parse or compare either value.
+ for key in ("cached_dttm", "cache_dttm"):
+ if key not in payload:
+ continue
+ cache_dttm = dict.__getitem__(payload, key)
+ if cache_dttm is None:
+ continue
+ if type(cache_dttm) is str:
+ if _bounded_utf8_length(cache_dttm, _MAX_CACHE_STRING_BYTES) is not None:
+ continue
+ return _malformed_result(f"{label} {key} must be a bounded exact string")
+ if type(cache_dttm) is datetime:
+ tzinfo = cache_dttm.tzinfo
+ if tzinfo is None or any(
+ type(tzinfo) is trusted for trusted in _TRUSTED_TZINFO_TYPES
+ ):
+ continue
+ return _malformed_result(f"{label} {key} has an unsupported timezone")
+ return _malformed_result(
+ f"{label} {key} must be a bounded exact string or datetime"
+ )
+ return None
+
+
+def _excel_temporal_cell(value: Any) -> Any | None:
+ """Return a naive builtin temporal value an XLSX writer stores natively.
+
+ Called only after ``_normalize_trusted_scalar`` accepted ``value``. XLSX has
+ no timezone-aware date cells, so aware values keep their ISO text, as do
+ NumPy durations whose unit may be ambiguous.
+ """
+ value_type = type(value)
+ if value_type is date or value_type is timedelta:
Review Comment:
Fixed in f4a4fb961902efbe4433de86c45bba6d431b6582. Verified the reported writer behavior. Pre-1900 dates/datetimes, negative durations (including pandas sub-microsecond values), and values beyond the supported serial/rounding range retain their existing ISO text. Representable values remain native cells.
Added regression coverage through the Excel export path for both openpyxl and xlsxwriter: all 36 out-of-range cases fail against the original helper and pass with the fix, with 20 valid-boundary cases also passing. Validation: chart suite 3,855 passed / 3 skipped; export/query-result tests 582 passed; pre-commit passed, including mypy, ruff, and pylint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5985047519
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4223874854
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1758,847 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if (
+ config.dimensions is None
+ and config._inherited_groupby is None
+ and config.order_by
+ ):
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
Review Comment:
Creating a Bullet chart without `row_limit` now saves no limit, even though `BulletChartConfig.row_limit` advertises a default of 10000 / max 50000 ("Maximum bullet rows"). On a deployment with `ROW_LIMIT=50000`, `generate_chart` with `dimensions=[{"name":"region"}]` and no `row_limit` over 15,000 regions stores a chart whose `get_chart_data` and previews return 15,000 rows instead of the documented 10,000; every other mapper in this file writes `row_limit` unconditionally. Can creation apply the schema default while updates keep preserving the saved value when the field is omitted?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2412,101 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ escape_separator(label)
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(label + FLAT_COLUMN_SEPARATOR)
Review Comment:
A saved grouped XY chart with the default `truncate_metric: true` (`metrics: ["revenue"]`, `groupby: ["region"]`, `x_axis: "day"`) comes back from the new query chain with the metric prefix already stripped (`chart_helpers.py` ~1380 renames the metric level to `None`), so rows look like `{day: ..., East: 10, West: 20}`. None of those keys start with `revenue` + the separator, so `fields` is empty, the adapter returns `None`, and the generic renderer plots a single numeric series with no legend while the preview still reports success. Should the field discovery also cover the truncated-metric shape, e.g. every non-x-axis key when `truncate_metric` is set and there is one metric?
##########
superset/mcp_service/chart/compile.py:
##########
@@ -116,14 +119,19 @@ def _compile_chart(
row_limit=plugin.compile_row_limit(form_data) if plugin else 2,
force=False,
)
+ set_query_context_form_data(query_context, dataset_id, "table")
command = ChartDataCommand(query_context)
command.validate()
result = command.run()
warnings: List[str] = []
row_count = 0
- if query_failure := query_result_failure(result):
+ query_data, query_failure = query_result_data(
Review Comment:
A grouped Gauge whose query returns one non-finite dial next to finite ones (say `[{team: "A", score: Infinity}, {team: "B", score: 42}]` from a Postgres division) now fails `generate_chart`/`update_chart` with `CHART_COMPILE_FAILED` ("contains a non-finite number"). This `query_result_data` call runs before `normalize_chart_query_result`, and without `preserve_nonfinite_floats=True` it rejects the infinity up front, so the Gauge normalizer that skips invalid dials and renders B never runs; before this change that result compiled. Should this call pass `preserve_nonfinite_floats=True` (as `normalize_treemap_query_result` does), with the same for the new preview validation calls?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4224615351
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,112 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ prefixes = {
+ spelling
+ for label in metric_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
+ field
+ for field in data[0]
+ if field != x_axis
+ and any(
+ field.startswith(prefix + FLAT_COLUMN_SEPARATOR)
+ or field.startswith(prefix + "__")
+ for prefix in prefixes
+ )
+ ]
+ if not fields and len(metric_labels) == 1 and form_data.get("truncate_metric"):
+ # A single truncated metric drops its label from the pivoted column
+ # names, so every non-x-axis column is one category series.
+ fields = [field for field in data[0] if field != x_axis]
+ if not fields:
+ return None
+ sample = data[0][x_axis]
+ x_type = (
+ "temporal"
+ if isinstance(sample, str) and any(char in sample for char in "-/: ")
+ else "quantitative"
+ if isinstance(sample, (int, float))
+ else "nominal"
+ )
+ return VegaLitePreview(
+ specification={
+ "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
+ "data": {"values": data},
+ "transform": [
+ {
+ "fold": [
+ "".join(
+ "\\" + char if char in ".[]\\" else char for char in field
+ )
+ for field in fields
+ ],
+ "as": ["__mcp_xy_series", "__mcp_xy_value"],
+ }
+ ],
+ "mark": mark,
+ "encoding": {
+ "x": {"field": x_axis, "type": x_type, "title": x_axis},
Review Comment:
The folded series fields are escaped for `.`, `[`, `]` and `\`, but the x encoding here (and the x field in the tooltip below) uses the raw `x_axis` name. For a chart whose x-axis column is `orders.date`, Vega-Lite reads that as a nested path, finds no value, and the x axis renders empty while the preview still reports success. Should the x references go through the same escaping?
##########
superset/mcp_service/chart/tool/update_chart_preview.py:
##########
@@ -208,11 +245,27 @@ def update_chart_preview( # noqa: C901
or (previous_form_data or {}).get("datasource_id")
or ""
).split("__", 1)[0]
- plugin = get_registry().get(config.chart_type, include_disabled=True)
+ # The cached chart keeps its plugin's contract for the whole
+ # update, even when its chart type is disabled for new charts.
+ contract_scope.enter_context(
+ saved_chart_contract((previous_form_data or {}).get("viz_type"))
+ )
+ plugin = get_registry().get(config.chart_type)
dataset_rebind = previous_datasource != str(dataset.id) and (
bool(previous_datasource)
or bool(plugin and plugin.unbound_form_data_is_rebind)
)
+ if (
+ dataset_rebind
+ and previous_form_data
+ and not (plugin is not None and plugin.strict_dataset_rebind)
+ ):
+ # Match saved-chart rebinds: retain only references that resolve
+ # against the replacement dataset before resolving omitted roles.
+ previous_form_data = _prune_inherited_query_state(
Review Comment:
This prune treats the rebind as same-dataset, but `_inherited_state_invalid_keys` (`update_chart.py` ~378) only checks `groupby` when it is a list, and Bullet explicitly supports a scalar saved `groupby` (`bullet_groupby_list` mirrors the frontend `ensureIsArray`). For a saved Bullet with `groupby: "OldRegion"` rebound to a dataset that has `Revenue` but no `OldRegion` (`update_chart_preview` with `dataset_id` and only a new metric), the stale hierarchy survives the prune, gets normalized to `["OldRegion"]`, and the query fails on a missing column instead of dropping the incompatible hierarchy as the docs describe. Should the scalar form be validated like the list form?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1398,6 +2413,112 @@ def fallback_vega_lite_preview(
return None
+def generate_xy_pivot_vega_lite_preview(
+ data: list[dict[str, Any]], form_data: dict[str, Any], *, mark: str
+) -> VegaLitePreview | None:
+ """Render flattened timeseries pivot columns without dropping grouped series.
+
+ Folding escaped field paths resolves literal output keys without splitting
+ category values that contain escaped commas. The legend retains each
+ complete metric/category label.
+ Long-form results continue through the generic renderer.
+ """
+ from superset.mcp_service.chart.chart_helpers import _as_list
+ from superset.utils.pandas_postprocessing.utils import (
+ escape_separator,
+ FLAT_COLUMN_SEPARATOR,
+ )
+
+ if not data:
+ return None
+ dimensions = [
+ label
+ for column in _as_list(form_data.get("groupby"))
+ if (label := _form_column_label(column))
+ ]
+ if not dimensions or any(label in data[0] for label in dimensions):
+ return None
+ x_axis = _form_column_label(form_data.get("x_axis")) or "__timestamp"
+ if x_axis not in data[0]:
+ return None
+ metric_labels = [
+ label
+ for metric in _as_list(form_data.get("metrics"))
+ if (label := metric_result_label(metric))
+ ]
+ # Chart-data results unescape the flattened column names, while raw
+ # post-processing output keeps escaped separators; match either spelling.
+ prefixes = {
+ spelling
+ for label in metric_labels
+ for spelling in (label, escape_separator(label))
+ }
+ fields = [
Review Comment:
A saved grouped line chart with one metric and a time comparison (say `metrics: ["revenue"]`, `groupby: ["region"]`, `time_compare: ["1 year ago"]`, `comparison_type: "values"`) comes back from the new rename step with keys like `revenue, East` and `1 year ago, East` (`chart_helpers.py` ~1372 renames the shifted series to the bare offset when there is one metric). This prefix filter only keeps fields that start with `revenue`, so the prior-year series is folded out silently and the preview still reports success. The `truncate_metric` fallback below only runs when no field matched, so it doesn't cover this mixed shape either. Should field discovery also accept the offset-named series (or fold every non-x-axis column when the shapes are mixed)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5930129219
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4148800220
##########
superset/mcp_service/chart/tool/get_chart_data.py:
##########
@@ -1441,10 +1420,26 @@ def _write_excel_headers(ws: Any, columns: List[str]) -> None:
ws.cell(row=1, column=idx, value=col)
+def _excel_scalar(value: Any) -> Any:
+ """Project a validated result scalar to values supported by Excel writers."""
+ if value is None:
+ return ""
+ if type(value) is UUID:
+ return UUID.__str__(value)
+ if type(value) is list or type(value) is dict:
+ return str(value)
+ return value
+
+
def _write_excel_data(ws: Any, data: List[Dict[str, Any]], columns: List[str]) -> None:
"""Write data to Excel worksheet."""
for row_idx, row in enumerate(data, 2):
for col_idx, col in enumerate(columns, 1):
+ ws.cell(
+ row=row_idx,
+ column=col_idx,
+ value=_excel_scalar(row.get(col, "")),
Review Comment:
This new write is immediately overwritten by the pre-existing `ws.cell(...)` call a few lines below, which recomputes `value` straight from `row.get(col, "")` without `_excel_scalar`'s UUID/non-finite-float/list/dict handling — so that handling never actually takes effect. A UUID column in the result makes `get_chart_data(format="excel")` raise instead of returning a workbook, since openpyxl can't serialize a raw `UUID`. Should the old write below be removed now that this one replaces it?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1373,15 +1406,32 @@ def _bind_dashboard_time_range_filter( # noqa: C901
if _uses_mapper_owned_temporal_binding(form_data, dataset_id):
return
+ explicit_fields = set(getattr(config, "model_fields_set", set()))
+ if _clears_temporal_subject(config, explicit_fields):
Review Comment:
This branch correctly skips generating a new temporal binding when `temporal_column` is explicitly `null`, but it never clears the existing `_mcp_dashboard_time_filter_subject` marker either — the key is just absent from this dict. The overlay merge in `_merge_shared_form_data` (`{**existing_form_data, **new_form_data}`) then re-inherits the old marker from the saved chart, so an explicit clear is silently reinterpreted as omission (or raises once the matching filter has already been dropped elsewhere). Should this branch explicitly set the marker to `None` so the merge step can tell "cleared" apart from "omitted"?
##########
superset/mcp_service/chart/chart_utils.py:
##########
@@ -1715,6 +1778,668 @@ def map_histogram_config(config: "HistogramChartConfig") -> Dict[str, Any]:
return form_data
+def _bullet_token_list(values: Sequence[str | int | float]) -> str:
+ """Serialize typed Bullet controls to the frontend's comma-separated form."""
+ tokens: list[str] = []
+ for value in values:
+ if isinstance(value, float):
+ token = repr(value)
+ # ``100`` parses back to the same binary float as ``100.0`` and
+ # preserves the frontend's established compact integer spelling.
+ if token.endswith(".0") and not (
+ value == 0.0 and math.copysign(1.0, value) < 0
+ ):
+ token = token[:-2]
+ tokens.append(token)
+ else:
+ tokens.append(str(value))
+ return ",".join(tokens)
+
+
+def map_bullet_config(config: BulletChartConfig) -> Dict[str, Any]: # noqa: C901
+ """Map typed Bullet config to ``Bullet/buildQuery`` and transformProps.
+
+ The frontend buildQuery replaces the generic query fields with exactly one
+ metric and the groupby hierarchy. Presentation controls stay in native
+ snake_case form_data; the chart plugin camelizes them for transformProps.
+ """
+ if config.dimensions is None and config.order_by:
+ # An update resolves its saved hierarchy before mapping. Without one,
+ # creation must validate sort targets against an empty hierarchy.
+ BulletChartConfig.model_validate(
+ {**config.model_dump(exclude_unset=True), "dimensions": []}
+ )
+ metric = create_metric_object(config.metric)
+ form_data: Dict[str, Any] = {
+ "viz_type": "bullet",
+ "metric": metric,
+ }
+
+ # Optional semantic/query fields are emitted only when explicitly supplied.
+ # This lets update_chart and update_chart_preview preserve native saved state,
+ # while an explicit empty value still clears it through the generic merge path.
+ if "dimensions" in config.model_fields_set:
+ form_data["groupby"] = [dimension.name for dimension in config.dimensions or []]
+ if "row_limit" in config.model_fields_set:
+ form_data["row_limit"] = config.row_limit
+ if "time_range" in config.model_fields_set:
+ form_data["time_range"] = config.time_range
+
+ if config.order_by:
+ dimensions = config.dimensions or []
+ orderby: list[list[Any]] = []
+ for order in config.order_by:
+ role, index = resolve_bullet_order_target(
+ order.column, dimensions, config.metric
+ )
+ if role == "metric":
+ order_target: Any = metric
+ else:
+ if index is None: # Defensive: resolver pairs dimensions with indexes.
+ raise ValueError("Bullet dimension order target has no index")
+ order_target = dimensions[index].name
+ orderby.append([order_target, order.ascending])
+ form_data["orderby"] = orderby
+ elif "order_by" in config.model_fields_set:
+ form_data["orderby"] = []
+
+ presentation_fields: dict[str, tuple[str, Any]] = {
+ "ranges": ("ranges", _bullet_token_list(config.ranges)),
+ "range_labels": (
+ "range_labels",
+ _bullet_token_list(config.range_labels),
+ ),
+ "markers": ("markers", _bullet_token_list(config.markers)),
+ "marker_labels": (
+ "marker_labels",
+ _bullet_token_list(config.marker_labels),
+ ),
+ "marker_lines": (
+ "marker_lines",
+ _bullet_token_list(config.marker_lines),
+ ),
+ "marker_line_labels": (
+ "marker_line_labels",
+ _bullet_token_list(config.marker_line_labels),
+ ),
+ "y_axis_format": ("y_axis_format", config.y_axis_format),
+ "show_labels": ("show_labels", config.show_labels),
+ "show_legend": ("show_legend", config.show_legend),
+ }
+ for field_name, (form_key, value) in presentation_fields.items():
+ if field_name in config.model_fields_set:
+ form_data[form_key] = value
+
+ _add_adhoc_filters(form_data, config.filters)
+ if config.filters == [] and "filters" in config.model_fields_set:
+ form_data["adhoc_filters"] = []
+ if config.time_range and config.temporal_column:
+ _ensure_temporal_adhoc_filter(form_data, config.temporal_column)
+ for filter_ in form_data.get("adhoc_filters", []):
+ if (
+ isinstance(filter_, dict)
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ and filter_.get("subject") == config.temporal_column
+ and filter_.get("comparator") == NO_TIME_RANGE
+ ):
+ filter_["comparator"] = config.time_range
+ return form_data
+
+
+def merge_bullet_form_data(
+ existing_form_data: Mapping[str, Any], new_form_data: Dict[str, Any]
+) -> None:
+ """Preserve omitted native Bullet controls across update tool paths.
+
+ Query roles and every UI control have an explicit typed representation.
+ Mappers emit optional fields only when the caller supplied them, so copying
+ the bounded native keys below preserves omitted state while explicit empty,
+ false, null, and zero-like values remain authoritative.
+ """
+ if (
+ existing_form_data.get("viz_type") != "bullet"
+ or new_form_data.get("viz_type") != "bullet"
+ ):
+ return
+ preserved_keys = {
+ "groupby",
+ "adhoc_filters",
+ "time_range",
+ "row_limit",
+ "orderby",
+ "ranges",
+ "range_labels",
+ "markers",
+ "marker_labels",
+ "marker_lines",
+ "marker_line_labels",
+ "y_axis_format",
+ "show_labels",
+ "show_legend",
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ }
+
+ # Threshold and label arrays are one frontend control pair. If callers
+ # replace the values without replacing their labels, clear the stale labels
+ # instead of accidentally reassigning them by position.
+ dependent_controls = {
+ "ranges": "range_labels",
+ "markers": "marker_labels",
+ "marker_lines": "marker_line_labels",
+ }
+ for values_key, labels_key in dependent_controls.items():
+ if values_key in new_form_data and labels_key not in new_form_data:
+ new_form_data[labels_key] = ""
+
+ for key in preserved_keys:
+ if (
+ key == MCP_DASHBOARD_TIME_FILTER_SUBJECT
+ and "adhoc_filters" in new_form_data
+ ):
+ # The marker describes a mapper-generated temporal filter. Do not
+ # retain stale provenance when an explicit filter update removed it.
+ continue
+ if key in existing_form_data and key not in new_form_data:
+ new_form_data[key] = existing_form_data[key]
+
+
+def _filter_identity(filter_: Any) -> tuple[Any, ...] | None:
+ """Return the native identity used when one filter replaces another."""
+ if not isinstance(filter_, Mapping):
+ return None
+ return (
+ filter_.get("clause"),
+ filter_.get("expressionType"),
+ filter_.get("subject"),
+ filter_.get("operator"),
+ )
+
+
+def _temporal_binding_filter(filters: list[Any], subject: Any) -> dict[str, Any] | None:
+ """Find the unique filter owned by a recorded MCP temporal marker."""
+ if subject is None:
+ return None
+ if not isinstance(subject, str) or not subject:
+ raise ValueError(
+ "MCP temporal binding provenance subject must be a non-empty string"
+ )
+ matches = [
+ filter_
+ for filter_ in filters
+ if isinstance(filter_, dict)
+ and filter_.get("subject") == subject
+ and filter_.get("operator") == FilterOperator.TEMPORAL_RANGE.value
+ ]
+ if len(matches) != 1:
Review Comment:
This raises whenever a chart's saved MCP temporal-binding marker (`_mcp_dashboard_time_filter_subject`) survives an update but its bound filter doesn't. For example, sending `filters: []` on a chart with an existing dashboard-time binding clears `adhoc_filters` (the shared merge's explicit-empty-clears-the-control path) without also clearing this marker, so this lookup lands here with `found 0`. A plain "remove this chart's filters" `update_chart`/`update_chart_preview` call now fails outright whenever a dashboard-time binding exists, instead of applying the clear. Should the marker be cleared alongside `adhoc_filters` in that case?
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -339,6 +407,532 @@ def _generate_safe_ascii_bar_chart(data: List[Dict[str, Any]]) -> str:
return "\n".join(lines)
+def _form_metric_label(metric: Any) -> str | None:
+ """Return the result-column label for a native QueryFormMetric."""
+ if type(metric) is str:
+ return metric
+ if type(metric) is not dict:
+ return None
+ if label := dict.get(metric, "label"):
+ return label if type(label) is str else None
+ if dict.get(metric, "expressionType") == "SQL":
+ expression = dict.get(metric, "sqlExpression")
+ return expression if type(expression) is str and expression else None
+ column = dict.get(metric, "column")
+ column_name = dict.get(column, "column_name") if type(column) is dict else column
+ aggregate = dict.get(metric, "aggregate")
+ if type(column_name) is str and type(aggregate) is str:
+ return f"{aggregate}({column_name})"
+ return None
+
+
+def _form_column_label(column: Any) -> str | None:
+ """Return the result-column label for a native QueryFormColumn."""
+ if type(column) is str:
+ return column
+ if type(column) is not dict:
+ return None
+ for key in ("label", "column_name"):
+ if type(value := dict.get(column, key)) is str and value:
+ return value
+ return None
+
+
+def _canonical_result_field(label: str | None, row: Dict[str, Any]) -> str | None:
+ """Resolve an exact or one unambiguous casefold result-field match."""
+ if label is None:
+ return None
+ if label in dict.keys(row):
+ return label
+ matches = [
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _require_result_field(label: str | None, row: dict[str, Any], role: str) -> str:
+ """Resolve a role without falling back to an unrelated result field."""
+ if not label:
+ raise BulletOutputError(f"Bullet {role} has no declared result alias")
+ if label in dict.keys(row):
+ return label
+ matches = sorted(
+ field
+ for field in dict.keys(row)
+ if type(field) is str and field.casefold() == label.casefold()
+ )
+ if len(matches) == 1:
+ return matches[0]
+ if matches:
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is ambiguous; candidates: "
+ f"{', '.join(matches)}"
+ )
+ raise BulletOutputError(
+ f"Bullet {role} alias {label!r} is missing from query output"
+ )
+
+
+def _safe_enum_backing(value: Any) -> Any:
+ """Extract Enum's stored value without public descriptors/conversions."""
+ value_type = type(value)
+ try:
+ mro = type.__getattribute__(value_type, "__mro__")
+ except (AttributeError, TypeError): # pragma: no cover - normal types have MRO
+ return value
+ if type(mro) is not tuple or not any(base is Enum for base in mro):
+ return value
+ try:
+ backing = object.__getattribute__(value, "_value_")
+ except Exception as ex:
+ raise BulletOutputError("Bullet output contains an unreadable enum") from ex
+ if not any(type(backing) is allowed for allowed in _ENUM_SCALAR_TYPES):
+ raise BulletOutputError("Bullet output contains an unsupported enum value")
+ return backing
+
+
+def _decimal_javascript_string(value: Decimal) -> str:
+ """Render an exact binary64 spelling with JavaScript Number thresholds."""
+ sign, digits_tuple, exponent = Decimal.as_tuple(value)
+ if type(exponent) is not int: # finite Decimals always have an integer exponent
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ if not any(digits_tuple):
+ return "0"
+
+ digits = "".join(str(digit) for digit in digits_tuple)
+ adjusted = len(digits) + exponent - 1
+ prefix = "-" if sign else ""
+ if -6 <= adjusted < 21:
+ point = len(digits) + exponent
+ if point <= 0:
+ text = f"0.{('0' * -point)}{digits}"
+ elif point >= len(digits):
+ text = digits + ("0" * (point - len(digits)))
+ else:
+ text = f"{digits[:point]}.{digits[point:]}"
+ if "." in text:
+ text = text.rstrip("0").rstrip(".")
+ return prefix + text
+
+ fraction = digits[1:].rstrip("0")
+ coefficient = digits[0] + (f".{fraction}" if fraction else "")
+ exponent_text = f"+{adjusted}" if adjusted >= 0 else str(adjusted)
+ return f"{prefix}{coefficient}e{exponent_text}"
+
+
+def _javascript_number_string(value: int | float | Decimal) -> str:
+ """Apply JSON-number -> IEEE-754 Number -> JavaScript String semantics.
+
+ Exact result scalars can retain precision that the frontend cannot: JSON
+ parsing first rounds a numeric token to binary64, and ``String`` then emits
+ the shortest round-tripping decimal with fixed notation for exponents in
+ [-6, 20]. Converting exact builtin scalars to an exact builtin float keeps
+ the path hook-free. Python and JavaScript use the same shortest
+ round-tripping binary64 digits; ``_decimal_javascript_string`` only adjusts
+ the notation thresholds and exponent spelling.
+
+ A finite integer or Decimal outside binary64's range becomes an infinity
+ after JSON parsing, matching JavaScript. Non-finite source values are
+ rejected by the trusted scalar normalizer before this helper is called.
+ """
+ value_type = type(value)
+ if value_type not in {int, float, Decimal}:
+ raise BulletOutputError("Bullet dimension contains an unsupported number")
+ if value_type is float and not math.isfinite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite number")
+ if isinstance(value, Decimal) and not Decimal.is_finite(value):
+ raise BulletOutputError("Bullet dimension contains a non-finite Decimal")
+ try:
+ number = float(value)
+ except OverflowError:
+ number = -math.inf if value < 0 else math.inf
+
+ if math.isinf(number):
+ return "-Infinity" if number < 0 else "Infinity"
+ if number == 0:
+ # String(-0) is "0" even though JSON.parse preserves negative zero.
+ return "0"
+ return _decimal_javascript_string(Decimal(float.__repr__(number)))
+
+
+def _bullet_category_value( # noqa: C901
+ value: Any, dimension: str, row_index: int
+) -> tuple[Any, str]:
+ """Return a JSON-safe value and bounded frontend ``String(value)`` text.
+
+ The trusted scalar normalizer is type-exact and does not dispatch through
+ application hooks. Vega data retains the normalized Chart Data wire value
+ (including epoch-ms temporal numbers); only the derived category key and
+ ASCII label use the JavaScript-compatible text.
+ """
+ from superset.mcp_service.chart.query_result import (
+ _bounded_utf8_length,
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+ _is_chart_data_duration_scalar,
+ _is_chart_data_temporal_scalar,
+ _normalize_trusted_scalar,
+ )
+
+ normalized: Any
+ reason: str | None
+ if _is_chart_data_temporal_scalar(value):
+ normalized, reason = _chart_data_temporal_number(value)
+ elif _is_chart_data_duration_scalar(value):
+ normalized, reason = _chart_data_duration_text(value)
+ else:
+ normalized, reason = _normalize_trusted_scalar(
+ value, max_string_bytes=_MAX_BULLET_TEXT_BYTES
+ )
+ if reason is not None:
+ if reason == "contains an unsupported or subclassed value":
+ reason = "has an unsupported value type"
+ elif "oversized string" in reason:
+ reason = "exceeds the size limit"
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} {reason}"
+ )
+
+ value_type = type(normalized)
+ if normalized is None:
+ text = "null"
+ elif value_type is str:
+ text = normalized
+ elif value_type is bool:
+ text = "true" if normalized else "false"
+ elif value_type is int or value_type is float or value_type is Decimal:
+ text = _javascript_number_string(normalized)
+ else:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} has an "
+ "unsupported value type"
+ )
+
+ if _bounded_utf8_length(text, _MAX_BULLET_TEXT_BYTES) is None:
+ raise BulletOutputError(
+ f"Bullet dimension {dimension!r} row {row_index} exceeds the size limit"
+ )
+ return normalized, text
+
+
+def _bullet_number(value: Any, row_index: int, metric_field: str) -> float:
+ """Apply the frontend's useful ``Number(value ?? 0)`` numeric subset."""
+ value = _safe_enum_backing(value)
+ if value is None:
+ number = 0.0
+ elif type(value) is bool:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned a boolean"
+ )
+ elif type(value) is int or type(value) is float or type(value) is Decimal:
+ try:
+ number = float(value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ ) from ex
+ elif type(value) is str:
+ if len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ stripped = value.strip()
+ if not stripped:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ try:
+ number = float(Decimal(stripped))
+ except (InvalidOperation, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} returned "
+ f"non-numeric text"
+ ) from ex
+ else:
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is not numeric"
+ )
+ if not math.isfinite(number):
+ raise BulletOutputError(
+ f"Bullet metric {metric_field!r} row {row_index} is NaN or infinite"
+ )
+ return number
+
+
+def _bullet_string_tokens(value: Any) -> list[str]:
+ """Parse labels exactly like the frontend's comma tokenizer."""
+ from superset.mcp_service.chart.query_result import _truncate_utf8
+
+ value = _safe_enum_backing(value)
+ if value is None:
+ return []
+ if type(value) is not str or len(value) > _MAX_BULLET_TEXT_BYTES:
+ raise BulletOutputError("Bullet labels must be a bounded comma-separated list")
+ if not value.strip():
+ return []
+ tokens = value.split(",")
+ if len(tokens) > _MAX_BULLET_TOKENS:
+ raise BulletOutputError("Bullet labels exceed the item limit")
+ return [_truncate_utf8(token.strip(), _MAX_BULLET_TEXT_BYTES) for token in tokens]
+
+
+def _unique_bullet_derived_field(
+ rows: list[dict[str, Any]], base: str, reserved: tuple[str, ...] = ()
+) -> str:
+ """Return one internal key absent from result rows and prior derived keys."""
+ occupied = {key for row in rows for key in dict.keys(row)}
+ occupied.update(reserved)
+ candidate = base
+ suffix = 0
+ while candidate in occupied:
+ suffix += 1
+ candidate = f"{base}_{suffix}"
+ return candidate
+
+
+def _unique_bullet_category_field(rows: list[dict[str, Any]]) -> str:
+ """Return an internal category key absent from every query-result row."""
+ return _unique_bullet_derived_field(rows, "__mcp_bullet_category")
+
+
+def _validate_bullet_format(format_: Any, values: list[float]) -> str:
+ """Reject a presentation format the backend cannot reproduce."""
+ format_ = _safe_enum_backing(format_)
+ if format_ is None or format_ == "":
+ format_ = "SMART_NUMBER"
+ if type(format_) is not str or len(format_) > 50:
+ raise BulletOutputError(
+ "Bullet number format is unsupported by previews",
+ error_type="UnsupportedFormat",
+ )
+ try:
+ for value in values:
+ _format_bullet_number(format_, value)
+ except (TypeError, ValueError, OverflowError) as ex:
+ raise BulletOutputError(
+ f"Bullet number format {format_!r} is unsupported by previews",
+ error_type="UnsupportedFormat",
+ ) from ex
+ return format_
+
+
+def _format_bullet_number(format_: str, value: float) -> str:
+ """Format finite Bullet values, including the full binary-float range."""
+ from superset.utils.number_format import format_numeric
+
+ try:
+ return format_numeric(format_, value)
+ except OverflowError:
+ # SMART_NUMBER's significant-digit rounding can overflow a finite float
+ # near DBL_MAX. Scientific repr remains deterministic and informative.
+ if format_ in {"SMART_NUMBER", "SMART_NUMBER_SIGNED"} and math.isfinite(value):
+ prefix = "+" if format_ == "SMART_NUMBER_SIGNED" and value > 0 else ""
+ return prefix + repr(value)
+ raise
+
+
+def _containing_bullet_range_label(
+ measure: float, ranges: list[float], labels: list[str]
+) -> str | None:
+ """Match the frontend's labelled containing-range tooltip selection."""
+ ascending = sorted(
+ (
+ (value, labels[index] if index < len(labels) else "")
+ for index, value in enumerate(ranges)
+ ),
+ key=lambda entry: entry[0],
+ )
+ for threshold, label in ascending:
+ if measure <= threshold:
+ return label or None
+ if ascending and ascending[-1][1]:
+ return f"> {ascending[-1][1]}"
+ return None
+
+
+def resolve_bullet_render_model( # noqa: C901
+ data: List[Dict[str, Any]], form_data: Dict[str, Any]
+) -> BulletRenderModel:
+ """Resolve and validate every Bullet row and presentation control."""
+ if type(data) is not list:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ for row_index in range(list.__len__(data)):
+ row = list.__getitem__(data, row_index)
+ if type(row) is not dict:
+ raise BulletOutputError("Bullet query output must be an array of objects")
+ if dict.__len__(row) > _MAX_BULLET_FIELDS:
+ raise BulletOutputError("Bullet query row exceeds the field limit")
+ for key in dict.keys(row):
+ if type(key) is not str:
+ raise BulletOutputError("Bullet query row keys must be strings")
+ if len(key) > _MAX_BULLET_FIELD_BYTES:
+ raise BulletOutputError("Bullet query row key exceeds the size limit")
+
+ if type(form_data) is not dict:
+ raise BulletOutputError("Bullet form data must be an object")
+
+ metric_label = _form_metric_label(dict.get(form_data, "metric"))
+ if not metric_label:
+ raise BulletOutputError("Bullet metric has no declared result alias")
+ raw_groupby = dict.get(form_data, "groupby")
+ if raw_groupby is None:
+ raw_groupby = []
+ if type(raw_groupby) is not list:
+ raise BulletOutputError("Bullet dimensions must be an array")
+ dimension_labels = [
+ _form_column_label(list.__getitem__(raw_groupby, index))
+ for index in range(list.__len__(raw_groupby))
+ ]
+ if any(not label for label in dimension_labels):
+ raise BulletOutputError("Bullet dimension has no declared result alias")
+
+ if data:
+ first_row = list.__getitem__(data, 0)
+ metric_field = _require_result_field(metric_label, first_row, "metric")
+ dimensions = [
+ _require_result_field(label, first_row, "dimension")
+ for label in dimension_labels
+ ]
+ else:
+ # The frontend accepts empty results. Ungrouped charts retain one
+ # zero-valued measure; grouped charts retain the declared roles but no
+ # categories or rows are fabricated.
+ metric_field = metric_label
+ dimensions = [label for label in dimension_labels if label is not None]
+
+ measures: list[float] = []
+ copied_rows: list[dict[str, Any]] = []
+ for index in range(list.__len__(data)):
+ row = list.__getitem__(data, index)
+ row_metric_field = _require_result_field(
+ metric_label, row, f"metric row {index}"
+ )
+ measure = _bullet_number(
+ dict.__getitem__(row, row_metric_field), index, metric_field
+ )
+ # Reserve every exact output key so the internal Vega category alias
+ # cannot collide with an unselected result field. Unselected values are
+ # deliberately replaced with None rather than converted or serialized.
+ copied: dict[str, Any] = dict.fromkeys(dict.keys(row))
+ copied[metric_field] = measure
+ for label, dimension in zip(dimension_labels, dimensions, strict=True):
+ row_dimension = _require_result_field(label, row, f"dimension row {index}")
+ dimension_value, _ = _bullet_category_value(
+ dict.__getitem__(row, row_dimension), dimension, index
+ )
+ copied[dimension] = dimension_value
+ copied_rows.append(copied)
+ measures.append(measure)
+
+ # The frontend validates/coerces the whole result array but renders only
+ # the first row for an ungrouped aggregate.
+ if not dimensions:
+ copied_rows = copied_rows[:1]
+ measures = measures[:1]
+ if not copied_rows:
+ copied_rows = [{metric_field: 0.0}]
+ measures = [0.0]
+
+ ranges = _strict_bullet_numeric_tokens(dict.get(form_data, "ranges"), "ranges")
Review Comment:
This raises `BulletOutputError` for any non-numeric token in `ranges`, but the ECharts frontend already tolerates and silently drops non-numeric tokens here when rendering a saved chart. A previously-saved Bullet chart with a stray non-numeric token in `ranges`/`markers`/`marker_lines` (which still renders fine in Explore) now fails every `get_chart_preview`/`get_chart_data` call through this path, and the native `List[float]` schema field rejects it on any unrelated `update_chart` too — the chart becomes permanently unreadable via MCP. Should this drop non-numeric tokens instead of raising, to match the frontend?
##########
superset/mcp_service/chart/validation/dataset_validator.py:
##########
@@ -524,6 +543,25 @@ def get_canonical_column_name(
The canonical column name from the dataset, or the original name
if no match is found.
"""
+ names = [col["name"] for col in dataset_context.available_columns]
+ names.extend(metric["name"] for metric in dataset_context.available_metrics)
Review Comment:
This builds one combined name list from `available_columns` and `available_metrics` before the case-fold check, so a physical column and a same-name-different-case saved metric now collide: with physical column `Revenue` and saved metric `REVENUE`, requesting `revenue` raises `AmbiguousDatasetReferenceError` instead of resolving to the physical column. Should physical columns keep precedence over same-cased metrics here, the way `resolve_dataset_column` orders them?
##########
superset/mcp_service/chart/plugins/bullet.py:
##########
@@ -0,0 +1,444 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""ECharts Bullet chart type plugin."""
+
+from __future__ import annotations
+
+from collections.abc import Callable, Mapping
+from typing import Any, ClassVar
+
+from superset.mcp_service.chart.chart_utils import (
+ _summarize_filters,
+ map_bullet_config,
+)
+from superset.mcp_service.chart.plugin import BaseChartPlugin
+from superset.mcp_service.chart.schemas import (
+ BulletChartConfig,
+ ChartError,
+ ColumnRef,
+ resolve_bullet_order_target,
+ VegaLitePreview,
+)
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+ is_numeric_column,
+ resolve_dataset_column,
+)
+from superset.mcp_service.common.error_schemas import (
+ ChartGenerationError,
+ DatasetContext,
+)
+
+
+def _canonical_reference(
+ name: str,
+ candidates: list[str],
+ role: str,
+) -> str:
+ """Resolve exact/casefold matches without silently choosing ambiguity."""
+ if name in candidates:
+ return name
+ matches = [
+ candidate for candidate in candidates if candidate.casefold() == name.casefold()
+ ]
+ if len(matches) > 1:
+ # AmbiguousDatasetReferenceError subclasses ValueError, so existing
+ # ValueError handlers keep working, while the validation pipeline
+ # re-raises it instead of downgrading it to a warning and proceeding
+ # with the unresolved reference.
+ raise AmbiguousDatasetReferenceError(name, matches, f"Bullet {role}")
+ return matches[0] if matches else name
+
+
+def _render_model_error(rows: Any, form_data: Mapping[str, Any]) -> ChartError | None:
+ """Return why ``rows`` cannot build the strict Bullet render model."""
+ from superset.mcp_service.chart.preview_utils import (
+ BulletOutputError,
+ resolve_bullet_render_model,
+ )
+ from superset.mcp_service.chart.query_result import safe_exception_message
+
+ try:
+ resolve_bullet_render_model(rows, dict(form_data))
+ except BulletOutputError as ex:
+ return ChartError(error=safe_exception_message(ex), error_type=ex.error_type)
+ return None
+
+
+class BulletChartPlugin(BaseChartPlugin):
+ """Plugin matching ``plugin-chart-echarts/src/Bullet``."""
+
+ chart_type = "bullet"
+ display_name = "Bullet Chart"
+ native_viz_types: ClassVar[Mapping[str, str]] = {
+ "bullet": "Bullet Chart",
+ }
+ # Bullet/transformProps.ts converts temporal values with Number().
+ temporal_json_numbers = True
+ # The frontend renders an empty result (a zero measure when ungrouped).
+ allows_empty_result = True
+ allows_empty_data_result = True
+ # Updates merge filter provenance plus Bullet's bounded native controls;
+ # no other saved control is carried into the typed Bullet state.
+ owns_update_merge = True
+ binds_time_range_to_temporal_filter = True
+ table_preview_unsupported_reason: ClassVar[str | None] = (
+ "Table previews cannot represent Bullet ranges, markers, "
+ "labels, and legend semantics"
+ )
+ invalid_result_error_code = "MALFORMED_BULLET_OUTPUT"
+ invalid_result_message = (
+ "Bullet query output does not contain a usable sizing measure."
+ )
+
+ def pre_validate(self, config: dict[str, Any]) -> ChartGenerationError | None:
+ if "metric" in config:
+ return None
+ return ChartGenerationError(
+ error_type="missing_bullet_fields",
+ message="Bullet chart missing required field: metric",
+ details=(
+ "A Bullet chart measures one numeric aggregate or saved/SQL metric; "
+ "optional dimensions split it into one row per group."
+ ),
+ suggestions=[
+ "Add metric: {'name': 'revenue', 'aggregate': 'SUM'}",
+ "For a saved metric use {'name': 'revenue', 'saved_metric': true}",
+ "Add dimensions: [{'name': 'region'}] for grouped bullet rows",
+ ],
+ error_code="MISSING_BULLET_FIELDS",
+ )
+
+ def extract_column_refs(self, config: Any) -> list[ColumnRef]:
+ if not isinstance(config, BulletChartConfig):
+ return []
+ refs = [config.metric, *(config.dimensions or [])]
+ refs.extend(ColumnRef(name=filter_.column) for filter_ in config.filters or [])
+ # order_by is constrained to role outputs by the schema, so those names
+ # are already represented by metric/dimension refs and must not be
+ # reinterpreted as physical columns.
+ return refs
+
+ def to_form_data(
+ self, config: Any, dataset_id: int | str | None = None
+ ) -> dict[str, Any]:
+ if not isinstance(config, BulletChartConfig):
+ raise TypeError("BulletChartPlugin requires BulletChartConfig")
+ return map_bullet_config(config)
+
+ def post_map_validate( # noqa: C901
+ self,
+ config: Any,
+ form_data: dict[str, Any],
+ dataset_id: int | str | None = None,
+ ) -> ChartGenerationError | None:
+ """Require an unambiguous numeric metric output for Number(...)."""
+ if not isinstance(config, BulletChartConfig) or dataset_id is None:
+ return None
+ dataset_context = DatasetValidator._get_dataset_context(dataset_id)
+ if dataset_context is None:
+ return None
+
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+ requested: list[tuple[str, list[str], str]] = []
+ if config.metric.name and not config.metric.sql_expression:
+ requested.append(
+ (
+ config.metric.name,
+ metrics if config.metric.saved_metric else columns,
+ "saved metric" if config.metric.saved_metric else "metric column",
+ )
+ )
+ requested.extend(
+ (dimension.name or "", columns, "dimension")
+ for dimension in config.dimensions or []
+ if dimension.name
+ )
+ requested.extend(
+ (filter_.column, columns, "filter column")
+ for filter_ in config.filters or []
+ )
+ if config.temporal_column:
+ requested.append((config.temporal_column, columns, "temporal column"))
+
+ for name, candidates, role in requested:
+ if (
+ name not in candidates
+ and sum(
+ candidate.casefold() == name.casefold() for candidate in candidates
+ )
+ > 1
+ ):
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet {role} {name!r} is ambiguous in dataset metadata"
+ ),
+ details=(
+ "Multiple dataset fields differ only by case. The query and "
+ "frontend require an exact canonical field name."
+ ),
+ suggestions=[
+ "Use get_dataset_info and copy the exact-case field name"
+ ],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+
+ metric = config.metric
+ if metric.saved_metric or metric.sql_expression:
+ # Saved/SQL metric result types are determined by their expressions;
+ # Tier-2 compile validation remains authoritative.
+ return None
+ if (metric.aggregate or "SUM") in {"COUNT", "COUNT_DISTINCT"}:
+ return None
+ try:
+ column = resolve_dataset_column(metric.name or "", dataset_context)
+ except ValueError as ex:
+ return ChartGenerationError(
+ error_type="ambiguous_bullet_reference",
+ message=(
+ f"Bullet metric column {metric.name!r} is ambiguous in "
+ "dataset metadata"
+ ),
+ details=str(ex),
+ suggestions=["Use get_dataset_info and copy the exact-case field name"],
+ error_code="AMBIGUOUS_BULLET_REFERENCE",
+ )
+ if column is None or is_numeric_column(column):
+ return None
+ return ChartGenerationError(
+ error_type="non_numeric_bullet_metric",
+ message=(
+ f"Bullet metric {metric.name!r} must produce a number; dataset "
+ f"type is {column.get('type', 'UNKNOWN')}."
+ ),
+ details=(
+ "Bullet/transformProps.ts converts the metric result with Number(). "
+ "A non-numeric MIN/MAX or default SUM would render an invalid bar."
+ ),
+ suggestions=[
+ "Use COUNT or COUNT_DISTINCT for a text column",
+ "Choose a numeric dataset column",
+ "Use a saved or SQL metric that returns a numeric value",
+ ],
+ error_code="NON_NUMERIC_BULLET_METRIC",
+ )
+
+ def normalize_column_refs(
+ self, config: Any, dataset_context: DatasetContext
+ ) -> Any:
+ if not isinstance(config, BulletChartConfig):
+ return config
+ explicit_fields = set(config.model_fields_set)
+ config_dict = config.model_dump(exclude_unset=True)
+ columns = [column["name"] for column in dataset_context.available_columns]
+ metrics = [metric["name"] for metric in dataset_context.available_metrics]
+
+ metric = config_dict["metric"]
+ if not metric.get("sql_expression"):
+ metric["name"] = _canonical_reference(
+ metric["name"],
+ metrics if metric.get("saved_metric") else columns,
+ "saved metric" if metric.get("saved_metric") else "metric column",
+ )
+ for dimension in config_dict.get("dimensions") or []:
+ dimension["name"] = _canonical_reference(
+ dimension["name"], columns, "dimension"
+ )
+ if temporal := config_dict.get("temporal_column"):
+ config_dict["temporal_column"] = _canonical_reference(
+ temporal, columns, "temporal column"
+ )
+ for filter_ in config_dict.get("filters") or []:
+ filter_["column"] = _canonical_reference(
+ filter_["column"], columns, "filter column"
+ )
+
+ # Sort targets may use ergonomic role names or labels. Canonicalize
+ # physical-name targets and leave explicit display labels untouched.
+ for order in config_dict.get("order_by") or []:
+ role, index = resolve_bullet_order_target(
+ order["column"], config.dimensions or [], config.metric
+ )
+ if role == "dimension" and index is not None:
+ order["column"] = config_dict["dimensions"][index]["name"]
+ elif not metric.get("sql_expression") and not metric.get("label"):
+ order["column"] = metric["name"]
+
+ normalized = BulletChartConfig.model_validate(config_dict)
+ normalized.model_fields_set.clear()
+ normalized.model_fields_set.update(explicit_fields)
+ return normalized
+
+ def generate_name(self, config: Any, dataset_name: str | None = None) -> str:
+ metric = config.metric.label or config.metric.name or "Metric"
+ what = f"{metric} bullet"
+ if config.dimensions:
+ what += " by " + ", ".join(
+ dimension.label or dimension.name or "dimension"
+ for dimension in config.dimensions
+ )
+ return self._with_context(what, _summarize_filters(config.filters))
+
+ def resolve_viz_type(self, config: Any) -> str:
+ return "bullet"
+
+ def schema_error_hint(self) -> ChartGenerationError | None:
+ return ChartGenerationError(
+ error_type="bullet_validation_error",
+ message="Bullet chart configuration validation failed",
+ details=(
+ "Bullet requires one numeric metric and optional unique physical "
+ "dimensions. Threshold/marker labels must align with their values."
Review Comment:
This hint is shown whenever Bullet schema validation fails for any reason, but "Threshold/marker labels must align with their values" isn't actually required — missing/short label arrays fall back to formatted numbers per index and extra labels are ignored. A caller whose config fails validation for an unrelated reason can be misled into "fixing" a label-count mismatch that was never a problem. Should this line be dropped or reworded?
##########
tests/unit_tests/mcp_service/chart/test_bullet_chart.py:
##########
@@ -0,0 +1,3786 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Product-path coverage for typed ECharts Bullet MCP support."""
+
+import math
+from datetime import date, datetime, time, timedelta, timezone, tzinfo
+from decimal import Decimal
+from enum import Enum, IntEnum, StrEnum
+from types import SimpleNamespace
+from typing import Any
+from unittest.mock import AsyncMock, MagicMock, patch
+from uuid import UUID
+from zoneinfo import ZoneInfo
+
+import numpy as np
+import pandas as pd
+import pytest
+import pytz
+from dateutil import tz as dateutil_tz
+from dateutil.zoneinfo import get_zonefile_instance
+from pydantic import TypeAdapter, ValidationError
+
+from superset.mcp_service.chart.chart_helpers import (
+ build_query_dicts_from_form_data,
+)
+from superset.mcp_service.chart.chart_utils import (
+ analyze_chart_capabilities,
+ map_bullet_config,
+ map_config_to_form_data,
+ MCP_DASHBOARD_TIME_FILTER_SUBJECT,
+ merge_update_form_data,
+ validate_merged_bullet_form_data,
+)
+from superset.mcp_service.chart.compile import _compile_chart
+from superset.mcp_service.chart.plugins.bullet import BulletChartPlugin
+from superset.mcp_service.chart.preview_utils import (
+ _generate_ascii_preview_from_data,
+ _generate_vega_lite_preview_from_data,
+ _javascript_number_string,
+ BulletOutputError,
+ generate_preview_from_form_data,
+ resolve_bullet_render_model,
+)
+from superset.mcp_service.chart.query_result import (
+ _chart_data_duration_text,
+ _chart_data_temporal_number,
+)
+from superset.mcp_service.chart.schemas import (
+ ASCIIPreview,
+ BulletChartConfig,
+ ChartConfig,
+ ChartError,
+ ChartInfo,
+ DataColumn,
+ GenerateChartRequest,
+ GetChartPreviewRequest,
+ UpdateChartPreviewRequest,
+ UpdateChartRequest,
+ VegaLitePreview,
+ XYChartConfig,
+)
+from superset.mcp_service.chart.tool.generate_chart import generate_chart
+from superset.mcp_service.chart.tool.get_chart_data import (
+ _candidates_single_numeric,
+ _VIZ_CATEGORY,
+)
+from superset.mcp_service.chart.tool.get_chart_preview import (
+ ASCIIPreviewStrategy,
+ TablePreviewStrategy,
+ VegaLitePreviewStrategy,
+)
+from superset.mcp_service.chart.tool.get_chart_type_schema import (
+ _get_chart_type_schema_impl,
+ VALID_CHART_TYPES,
+)
+from superset.mcp_service.chart.tool.update_chart import (
+ _build_preview_form_data,
+ _build_update_payload,
+ update_chart,
+)
+from superset.mcp_service.chart.tool.update_chart_preview import update_chart_preview
+from superset.mcp_service.chart.validation.dataset_validator import (
+ AmbiguousDatasetReferenceError,
+ DatasetValidator,
+)
+from superset.mcp_service.chart.validation.pipeline import ValidationPipeline
+from superset.mcp_service.common.error_schemas import DatasetContext
+from superset.utils.json import json_int_dttm_ser
+
+
+def _reject_scalar_conversion(*_args: object, **_kwargs: object) -> Any:
+ raise AssertionError("hostile query scalar method must not run")
+
+
+class _PathHostileStr(str):
+ __getitem__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _PathHostileEnum(str, Enum):
+ FAILED = "warehouse unavailable"
+
+ @property
+ def value(self) -> str:
+ """Reject the public descriptor while preserving Enum's stored value."""
+ return _reject_scalar_conversion()
+
+ __getitem__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileInt(int):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileFloat(float):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputHostileStr(str):
+ __str__ = _reject_scalar_conversion
+ strip = _reject_scalar_conversion
+
+
+class _OutputHostileDecimal(Decimal):
+ __float__ = _reject_scalar_conversion
+ __str__ = _reject_scalar_conversion
+
+
+class _OutputSafeIntEnum(IntEnum):
+ VALUE = 12
+
+
+class _OutputSafeStrEnum(StrEnum):
+ VALUE = "12.5"
+
+
+_OutputSafeIntEnum.__float__ = _reject_scalar_conversion # type: ignore[method-assign]
+_OutputSafeIntEnum.__str__ = _reject_scalar_conversion # type: ignore[method-assign]
+_OutputSafeStrEnum.__float__ = _reject_scalar_conversion # type: ignore[attr-defined]
+_OutputSafeStrEnum.__str__ = _reject_scalar_conversion # type: ignore[method-assign]
+
+
+def _simple_metric(name: str = "revenue") -> dict[str, str]:
+ return {"name": name, "aggregate": "SUM"}
+
+
+def _tool_user() -> SimpleNamespace:
+ return SimpleNamespace(id=1, username="admin", roles=[], groups=[])
+
+
+def _orm_dataset() -> SimpleNamespace:
+ def column(
+ name: str, type_: str, *, temporal: bool = False, numeric: bool = False
+ ) -> SimpleNamespace:
+ return SimpleNamespace(
+ column_name=name,
+ type=type_,
+ is_temporal=temporal,
+ is_numeric=numeric,
+ is_dttm=temporal,
+ python_date_format=None,
+ )
+
+ return SimpleNamespace(
+ id=7,
+ table_name="sales",
+ schema=None,
+ main_dttm_col="OrderDate",
+ database=SimpleNamespace(database_name="main", db_engine_spec=None),
+ columns=[
+ column("Revenue", "NUMERIC", numeric=True),
+ column("Region", "VARCHAR"),
+ column("Team", "VARCHAR"),
+ column("Status", "VARCHAR"),
+ column("OrderDate", "TIMESTAMP", temporal=True),
+ column("EventDate", "TIMESTAMP", temporal=True),
+ ],
+ metrics=[
+ SimpleNamespace(
+ metric_name="SavedRevenue",
+ expression="SUM(Revenue)",
+ description=None,
+ )
+ ],
+ )
+
+
+def test_bullet_discriminated_union_uses_exact_tag() -> None:
+ config = TypeAdapter(ChartConfig).validate_python(
+ {"chart_type": "bullet", "metric": _simple_metric()}
+ )
+ assert isinstance(config, BulletChartConfig)
+ with pytest.raises(ValidationError):
+ TypeAdapter(ChartConfig).validate_python(
+ {"chart_type": "bullet_chart", "metric": _simple_metric()}
+ )
+
+
+def test_bullet_equal_dimension_aliases_are_order_independent_and_round_trip() -> None:
+ for payload in (
+ {
+ "dimensions": [{"name": "Region", "label": "Market"}, "Team"],
+ "groupby": ["Region", {"column_name": "Team"}],
+ },
+ {
+ "groupby": ["Region", {"column": "Team"}],
+ "dimensions": [{"name": "Region", "label": "Market"}, "Team"],
+ },
+ ):
+ config = BulletChartConfig.model_validate(
+ {"metric": _simple_metric(), **payload}
+ )
+ assert [dimension.name for dimension in config.dimensions or []] == [
+ "Region",
+ "Team",
+ ]
+ mapped = map_bullet_config(config)
+ assert mapped["groupby"] == ["Region", "Team"]
+ round_trip = BulletChartConfig.model_validate(mapped)
+ assert [dimension.name for dimension in round_trip.dimensions or []] == [
+ "Region",
+ "Team",
+ ]
+
+
+def test_bullet_dimension_aliases_require_exact_physical_identity() -> None:
+ """Schema-only alias resolution must not merge quoted case variants."""
+ with pytest.raises(ValidationError, match="Conflicting Bullet dimension aliases"):
+ BulletChartConfig.model_validate(
+ {
+ "metric": _simple_metric(),
+ "dimensions": ["Region"],
+ "groupby": ["region"],
+ }
+ )
+
+
+@pytest.mark.parametrize(
+ "request_payload",
+ [
+ {"dataset_id": 7},
+ {"identifier": 9},
+ {"dataset_id": 7, "form_data_key": "preview"},
+ ],
+)
+@pytest.mark.parametrize("reverse", [False, True])
+def test_bullet_request_models_reject_conflicting_dimension_aliases(
+ request_payload: dict[str, object], reverse: bool
+) -> None:
+ aliases = [
+ ("dimensions", [{"name": "Region"}, {"name": "Team"}]),
+ ("groupby", ["Team", "Region"]),
+ ]
+ if reverse:
+ aliases.reverse()
+ config = {"chart_type": "bullet", "metric": _simple_metric(), **dict(aliases)}
+ payload = {**request_payload, "config": config}
+ request_type = (
+ UpdateChartRequest
+ if "identifier" in request_payload
+ else (
+ UpdateChartPreviewRequest
+ if "form_data_key" in request_payload
+ else GenerateChartRequest
+ )
+ )
+ with pytest.raises(ValidationError, match="Conflicting Bullet dimension aliases"):
+ request_type.model_validate(payload)
+
+
+@pytest.mark.parametrize(
+ "metric",
+ [
+ {"name": "revenue", "aggregate": "SUM", "label": "Revenue"},
+ {"name": "saved_revenue", "saved_metric": True},
+ {"sql_expression": "SUM(revenue) / COUNT(*)", "label": "Average"},
+ ],
+)
+def test_bullet_accepts_simple_saved_and_sql_metrics(metric: dict[str, object]) -> None:
+ config = BulletChartConfig(metric=metric)
+ form_data = map_bullet_config(config)
+ assert form_data["viz_type"] == "bullet"
+ assert form_data["metric"]
+
+
+def test_bullet_native_form_data_round_trip_is_semantically_stable() -> None:
+ native = {
+ "viz_type": "bullet",
+ "datasource": "7__table",
+ "metric": {
+ "aggregate": "SUM",
+ "column": {"column_name": "Revenue"},
+ "expressionType": "SIMPLE",
+ "label": "Total Revenue",
+ },
+ "groupby": ["Region", "Team"],
+ "ranges": "100,250,500",
+ "range_labels": "Minimum,Target,Stretch",
+ "markers": "300",
+ "marker_labels": "Plan",
+ "marker_lines": "400",
+ "marker_line_labels": "Forecast",
+ "y_axis_format": "$,.0f",
+ "show_labels": False,
+ "show_legend": True,
+ "row_limit": 250,
+ "orderby": [["Region", True], ["Total Revenue", False]],
+ "adhoc_filters": [
+ {
+ "clause": "WHERE",
+ "expressionType": "SIMPLE",
+ "subject": "Status",
+ "operator": "==",
+ "comparator": "Active",
+ }
+ ],
+ }
+ config = BulletChartConfig.model_validate(native)
+ mapped = map_bullet_config(config)
+
+ assert mapped["metric"]["label"] == "Total Revenue"
+ assert mapped["groupby"] == ["Region", "Team"]
+ assert mapped["ranges"] == "100,250,500"
+ assert mapped["range_labels"] == "Minimum,Target,Stretch"
+ assert mapped["markers"] == "300"
+ assert mapped["marker_lines"] == "400"
+ assert mapped["show_labels"] is False
+ assert mapped["show_legend"] is True
+ assert mapped["orderby"][0] == ["Region", True]
+ assert mapped["orderby"][1][0]["label"] == "Total Revenue"
+ assert mapped["adhoc_filters"][0]["subject"] == "Status"
+
+
+def test_bullet_presentation_numbers_use_shortest_round_trip_safe_tokens() -> None:
+ ranges = [1.2345678901234567, 1.7976931348623157e308]
+ markers = [5e-324, -0.0]
+ marker_lines = [9.876543210987654e-200]
+ config = BulletChartConfig(
+ metric=_simple_metric(),
+ ranges=ranges,
+ markers=markers,
+ marker_lines=marker_lines,
+ show_legend=True,
+ )
+ mapped = map_bullet_config(config)
+
+ for key, expected in (
+ ("ranges", ranges),
+ ("markers", markers),
+ ("marker_lines", marker_lines),
+ ):
+ tokens = mapped[key].split(",")
+ assert [float(token) for token in tokens] == expected
+ assert all(
+ float(token).hex() == value.hex()
+ for token, value in zip(tokens, expected, strict=True)
+ )
+
+ round_trip = BulletChartConfig.model_validate(mapped)
+ assert round_trip.ranges == ranges
+ assert round_trip.markers == markers
+ assert round_trip.marker_lines == marker_lines
+
+ model = resolve_bullet_render_model(
+ [{"SUM(revenue)": 1.0}],
+ mapped,
+ )
+ assert model.ranges == ranges
+ assert model.markers == markers
+ assert model.marker_lines == marker_lines
+ assert (
+ "1.7976931348623157e+308"
+ in _generate_ascii_preview_from_data(
+ [{"SUM(revenue)": 1.0}], mapped
+ ).ascii_content
+ )
+ vega = _generate_vega_lite_preview_from_data([{"SUM(revenue)": 1.0}], mapped)
+ assert vega.specification["layer"]
+
+
+def test_bullet_native_saved_metric_and_legacy_metric_aliases() -> None:
+ saved = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": "saved_revenue"}
+ )
+ legacy = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": "sum__revenue"}
+ )
+ assert saved.metric.saved_metric is True
+ assert saved.metric.name == "saved_revenue"
+ assert legacy.metric.name == "sum__revenue"
+ assert legacy.metric.saved_metric is True
+
+
+@pytest.mark.parametrize("metric_name", ["sum__num", "sum__SP_POP_TOTL"])
+@pytest.mark.parametrize(
+ ("request_type", "request_fields"),
+ [
+ (GenerateChartRequest, {"dataset_id": 7}),
+ (UpdateChartRequest, {"identifier": 9}),
+ (UpdateChartPreviewRequest, {"dataset_id": 7}),
+ ],
+)
+def test_bullet_repository_metric_names_round_trip_as_saved_metrics_on_all_requests(
+ metric_name: str,
+ request_type: type[
+ GenerateChartRequest | UpdateChartRequest | UpdateChartPreviewRequest
+ ],
+ request_fields: dict[str, object],
+) -> None:
+ request = request_type.model_validate(
+ {**request_fields, "config": {"chart_type": "bullet", "metric": metric_name}}
+ )
+ config = request.config
+ assert isinstance(config, BulletChartConfig)
+ assert config.metric.saved_metric is True
+ assert map_bullet_config(config)["metric"] == metric_name
+
+
+@pytest.mark.parametrize(
+ ("native_operator", "typed_operator", "round_trip_operator"),
+ [
+ ("EQUALS", "=", "=="),
+ ("NOT_EQUALS", "!=", "!="),
+ ("LESS_THAN", "<", "<"),
+ ("LESS_THAN_OR_EQUAL", "<=", "<="),
+ ("GREATER_THAN", ">", ">"),
+ ("GREATER_THAN_OR_EQUAL", ">=", ">="),
+ ("IN", "IN", "IN"),
+ ("NOT_IN", "NOT IN", "NOT IN"),
+ ("LIKE", "LIKE", "LIKE"),
+ ("ILIKE", "ILIKE", "ILIKE"),
+ ("IS_NULL", "IS NULL", "IS NULL"),
+ ("IS_NOT_NULL", "IS NOT NULL", "IS NOT NULL"),
+ ],
+)
+def test_bullet_native_filter_operator_names_round_trip(
+ native_operator: str, typed_operator: str, round_trip_operator: str
+) -> None:
+ comparator: object
+ if native_operator.startswith("IS_"):
+ comparator = None
+ elif native_operator in {"IN", "NOT_IN"}:
+ comparator = ["North"]
+ else:
+ comparator = "North"
+ config = BulletChartConfig.model_validate(
+ {
+ "viz_type": "bullet",
+ "metric": "SavedRevenue",
+ "adhoc_filters": [
+ {
+ "clause": "WHERE",
+ "expressionType": "SIMPLE",
+ "subject": "Region",
+ "operator": native_operator,
+ "comparator": comparator,
+ }
+ ],
+ }
+ )
+
+ assert config.filters is not None
+ assert config.filters[0].op == typed_operator
+ mapped = map_bullet_config(config)
+ assert mapped["adhoc_filters"][0]["operator"] == round_trip_operator
+
+
+def test_bullet_legacy_label_only_saved_metric_adapter_is_strict_and_bounded() -> None:
+ config = BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": {"label": "sum__num"}}
+ )
+ assert config.metric.saved_metric is True
+ assert map_bullet_config(config)["metric"] == "sum__num"
+
+ with pytest.raises(ValidationError):
+ BulletChartConfig.model_validate(
+ {
+ "viz_type": "bullet",
+ "metric": {"label": "sum__num", "aggregate": "SUM"},
+ }
+ )
+ with pytest.raises(ValidationError, match="at most 255"):
+ BulletChartConfig.model_validate(
+ {"viz_type": "bullet", "metric": {"label": "m" * 256}}
+ )
+
+
+@pytest.mark.parametrize(
+ "metric",
+ [
+ "SavedRevenue",
+ {
+ "aggregate": "SUM",
+ "column": {"column_name": "Revenue"},
+ "expressionType": "SIMPLE",
+ "label": "Simple Revenue",
+ },
+ {
+ "aggregate": None,
+ "column": None,
+ "expressionType": "SQL",
+ "sqlExpression": "SUM(Revenue)",
+ "label": "SQL Revenue",
+ },
+ ],
+)
+def test_bullet_all_metric_shapes_round_trip_full_native_presentation(
+ metric: object,
+) -> None:
+ native = {
+ "viz_type": "bullet",
+ "metric": metric,
+ "groupby": ["Region"],
+ "ranges": "50,100",
+ "range_labels": "Low,High",
+ "markers": "75",
+ "marker_labels": "Plan",
+ "marker_lines": "90",
+ "marker_line_labels": "Forecast",
+ "y_axis_format": "$,.0f",
+ "show_labels": True,
+ "show_legend": True,
+ }
+ mapped = map_bullet_config(BulletChartConfig.model_validate(native))
+ assert validate_merged_bullet_form_data(mapped) is not None
+ assert mapped["groupby"] == ["Region"]
+ assert mapped["ranges"] == "50,100"
+ assert mapped["marker_line_labels"] == "Forecast"
+
+
+def test_bullet_rejects_invalid_roles_and_output_collisions() -> None:
+ with pytest.raises(ValidationError, match="physical dimension"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[{"name": "region", "aggregate": "COUNT"}],
+ )
+ with pytest.raises(ValidationError, match="Duplicate Bullet dimension"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[{"name": "Region"}, {"name": "Region"}],
+ )
+ with pytest.raises(ValidationError, match="conflicts with a dimension"):
+ BulletChartConfig(
+ metric={"name": "revenue", "aggregate": "SUM", "label": "Region"},
+ dimensions=[{"name": "Region", "label": "Friendly"}],
+ )
+
+
+def test_bullet_accepts_short_labels_and_rejects_bad_order_target() -> None:
+ config = BulletChartConfig(
+ metric=_simple_metric(), ranges=[1, 2], range_labels=["Only one"]
+ )
+ assert config.range_labels == ["Only one"]
+ with pytest.raises(ValidationError, match="unknown: not_a_role"):
+ BulletChartConfig(
+ metric=_simple_metric(), dimensions=[], order_by=[{"column": "not_a_role"}]
+ )
+
+
+def test_bullet_dimension_labels_are_input_aliases_not_result_aliases() -> None:
+ config = BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Total"},
+ dimensions=[
+ {"name": "Team", "label": "Region"},
+ {"name": "Region", "label": "Market"},
+ ],
+ # The exact physical Region must win over Team's display label.
+ order_by=[
+ {"column": "Region", "ascending": True},
+ {"column": "Revenue", "ascending": False},
+ ],
+ )
+ form_data = map_bullet_config(config)
+ assert form_data["groupby"] == ["Team", "Region"]
+ assert form_data["orderby"] == [
+ ["Region", True],
+ [form_data["metric"], False],
+ ]
+
+ label_order = map_bullet_config(
+ BulletChartConfig(
+ metric=config.metric,
+ dimensions=config.dimensions,
+ order_by=[{"column": "Market"}],
+ )
+ )
+ assert label_order["orderby"] == [["Region", False]]
+
+ model = resolve_bullet_render_model(
+ [{"Team": "Blue", "Region": "North", "Total": 10}], form_data
+ )
+ assert model.dimensions == ["Team", "Region"]
+ assert [model.rows[0][name] for name in model.dimensions] == ["Blue", "North"]
+
+
+def test_bullet_rejects_ambiguous_display_alias_for_ordering() -> None:
+ with pytest.raises(ValidationError, match="ambiguous display alias"):
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[
+ {"name": "Region", "label": "Area"},
+ {"name": "Team", "label": "area"},
+ ],
+ order_by=[{"column": "AREA"}],
+ )
+
+
+@pytest.mark.parametrize(
+ ("metric", "order_target", "output"),
+ [
+ (
+ {"name": "SavedRevenue", "saved_metric": True, "label": "Friendly"},
+ "Friendly",
+ "SavedRevenue",
+ ),
+ (
+ {"name": "Revenue", "aggregate": "SUM", "label": "Simple Total"},
+ "Revenue",
+ "Simple Total",
+ ),
+ (
+ {"sql_expression": "SUM(Revenue)", "label": "SQL Total"},
+ "SQL Total",
+ "SQL Total",
+ ),
+ ],
+)
+def test_bullet_metric_shapes_share_physical_dimension_output_contract(
+ metric: dict[str, object], order_target: str, output: str
+) -> None:
+ config = BulletChartConfig(
+ metric=metric,
+ dimensions=[{"name": "Region", "label": "Market"}],
+ order_by=[{"column": order_target}],
+ )
+ form_data = map_bullet_config(config)
+ assert form_data["groupby"] == ["Region"]
+ assert form_data["orderby"] == [[form_data["metric"], False]]
+ model = resolve_bullet_render_model([{"Region": "North", output: 12}], form_data)
+ assert model.metric_field == output
+ assert model.dimensions == ["Region"]
+
+
+def test_bullet_mapper_preserves_omission_and_honors_explicit_values() -> None:
+ omitted = map_bullet_config(BulletChartConfig(metric=_simple_metric()))
+ explicit = map_bullet_config(
+ BulletChartConfig(
+ metric=_simple_metric(),
+ dimensions=[],
+ filters=[],
+ ranges=[],
+ show_labels=False,
+ show_legend=False,
+ row_limit=42,
+ time_range=None,
+ )
+ )
+ for key in (
+ "groupby",
+ "adhoc_filters",
+ "ranges",
+ "show_labels",
+ "show_legend",
+ "row_limit",
+ "time_range",
+ ):
+ assert key not in omitted
+ assert explicit["groupby"] == []
+ assert explicit["adhoc_filters"] == []
+ assert explicit["ranges"] == ""
+ assert explicit["show_labels"] is False
+ assert explicit["show_legend"] is False
+ assert explicit["row_limit"] == 42
+ assert explicit["time_range"] is None
+
+
+def test_bullet_registry_schema_and_recommendation_metadata() -> None:
+ from superset.mcp_service.app import get_default_instructions
+ from superset.mcp_service.chart.registry import display_name_for_viz_type, get
+
+ plugin = get("bullet")
+ assert plugin is not None
+ assert plugin.resolve_viz_type(None) == "bullet"
+ assert display_name_for_viz_type("bullet") == "Bullet Chart"
+ assert "bullet" in VALID_CHART_TYPES
+ discovered = _get_chart_type_schema_impl("bullet")
+ assert discovered["chart_type"] == "bullet"
+ assert discovered["examples"][0]["ranges"] == [100000, 250000, 500000]
+ assert _VIZ_CATEGORY["bullet"] == "bullet"
+ candidates = _candidates_single_numeric(
+ DataColumn(
+ name="Revenue",
+ display_name="Revenue",
+ data_type="numeric",
+ sample_values=[1],
+ null_count=0,
+ unique_count=1,
+ ),
+ row_count=1,
+ )
+ assert "bullet chart" in candidates
+ guidance = get_default_instructions()
+ assert 'chart_type="bullet": Bullet Chart' in guidance
+ assert "waterfall, gantt, bullet, bubble_v2, and interactive_pivot" in guidance
+
+
+def test_bullet_dataset_normalization_canonicalizes_every_reference() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "Region", "type": "VARCHAR"},
+ {"name": "OrderDate", "type": "TIMESTAMP", "is_temporal": True},
+ {"name": "Status", "type": "VARCHAR"},
+ ],
+ available_metrics=[],
+ )
+ config = BulletChartConfig(
+ metric={"name": "revenue", "aggregate": "SUM"},
+ dimensions=[{"name": "region"}],
+ temporal_column="orderdate",
+ filters=[{"column": "status", "op": "=", "value": "active"}],
+ order_by=[{"column": "region", "ascending": True}],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ normalized = plugin.normalize_column_refs(config, context)
+ assert normalized.metric.name == "Revenue"
+ assert normalized.dimensions[0].name == "Region"
+ assert normalized.temporal_column == "OrderDate"
+ assert normalized.filters[0].column == "Status"
+ assert normalized.order_by[0].column == "Region"
+ assert normalized.model_fields_set == config.model_fields_set
+
+
+def test_bullet_dataset_normalization_rejects_ambiguous_casefold_candidates() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "revenue", "type": "NUMERIC", "is_numeric": True},
+ ],
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ config = BulletChartConfig(metric={"name": "REVENUE", "aggregate": "SUM"})
+ with pytest.raises(ValueError, match="Revenue, revenue"):
+ plugin.normalize_column_refs(config, context)
+
+
+def test_bullet_numeric_output_constraint_rejects_text_min() -> None:
+ from superset.mcp_service.chart.registry import get
+
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=[{"name": "status", "type": "VARCHAR"}],
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+ config = BulletChartConfig(metric={"name": "status", "aggregate": "MIN"})
+ with patch.object(DatasetValidator, "_get_dataset_context", return_value=context):
+ error = plugin.post_map_validate(config, {}, dataset_id=7)
+ assert error is not None
+ assert error.error_type == "non_numeric_bullet_metric"
+
+
+@pytest.mark.parametrize("reverse_metadata", [False, True])
+def test_bullet_exact_case_type_and_role_resolution_is_order_independent(
+ reverse_metadata: bool,
+) -> None:
+ from superset.mcp_service.chart.registry import get
+
+ columns = [
+ {"name": "Revenue", "type": "NUMERIC", "is_numeric": True},
+ {"name": "revenue", "type": "VARCHAR", "is_numeric": False},
+ {"name": "Region", "type": "VARCHAR"},
+ ]
+ if reverse_metadata:
+ columns.reverse()
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=columns,
+ available_metrics=[],
+ )
+ plugin = get("bullet")
+ assert plugin is not None
+
+ numeric = BulletChartConfig(metric={"name": "Revenue", "aggregate": "SUM"})
+ text = BulletChartConfig(metric={"name": "revenue", "aggregate": "MIN"})
+ with patch.object(DatasetValidator, "_get_dataset_context", return_value=context):
+ assert plugin.post_map_validate(numeric, {}, dataset_id=7) is None
+ error = plugin.post_map_validate(text, {}, dataset_id=7)
+ assert error is not None
+ assert error.error_type == "non_numeric_bullet_metric"
+
+ roles = BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM"},
+ dimensions=[{"name": "revenue"}, {"name": "Region"}],
+ filters=[{"column": "revenue", "op": "=", "value": "retail"}],
+ order_by=[{"column": "revenue", "ascending": True}],
+ )
+ normalized = plugin.normalize_column_refs(roles, context)
+ assert normalized.metric.name == "Revenue"
+ assert [dimension.name for dimension in normalized.dimensions or []] == [
+ "revenue",
+ "Region",
+ ]
+ assert normalized.filters
+ assert normalized.filters[0].column == "revenue"
+ assert normalized.order_by[0].column == "revenue"
+
+ ambiguous = BulletChartConfig(metric={"name": "REVENUE", "aggregate": "SUM"})
+ with pytest.raises(
+ AmbiguousDatasetReferenceError, match="Bullet metric column reference"
+ ):
+ plugin.normalize_column_refs(ambiguous, context)
+
+
+@pytest.mark.parametrize("reverse_metadata", [False, True])
+def test_generic_aggregation_validation_uses_exact_case_before_type(
+ reverse_metadata: bool,
+) -> None:
+ from superset.mcp_service.chart.schemas import PieChartConfig
+
+ columns = [
+ {"name": "Amount", "type": "BIGINT", "is_numeric": True},
+ {"name": "amount", "type": "VARCHAR", "is_numeric": False},
+ ]
+ if reverse_metadata:
+ columns.reverse()
+ context = DatasetContext(
+ id=7,
+ table_name="sales",
+ schema=None,
+ database_name="main",
+ available_columns=columns,
+ available_metrics=[],
+ )
+
+ assert (
+ DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "Amount", "aggregate": "SUM"}).metric],
+ context,
+ )
+ == []
+ )
+ errors = DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "amount", "aggregate": "SUM"}).metric],
+ context,
+ )
+ assert errors
+ assert errors[0].error_type == "invalid_aggregation"
+
+ ambiguous = DatasetValidator._validate_aggregations(
+ [BulletChartConfig(metric={"name": "AMOUNT", "aggregate": "SUM"}).metric],
+ context,
+ )
+ assert ambiguous
+ assert ambiguous[0].error_type == "ambiguous_column_reference"
+
+ valid, error = DatasetValidator.validate_against_dataset(
+ PieChartConfig(
+ dimension={"name": "amount"},
+ metric={"name": "Amount", "aggregate": "SUM"},
+ ),
+ 7,
+ dataset_context=context,
+ )
+ assert valid is True
+ assert error is None
+
+
+@pytest.mark.parametrize(
+ ("metric", "field"),
+ [
+ ({"name": "Revenue", "aggregate": "SUM", "label": "Simple"}, "Simple"),
+ ({"name": "SavedRevenue", "saved_metric": True}, "SavedRevenue"),
+ ({"sql_expression": "SUM(Revenue)", "label": "SQL Total"}, "SQL Total"),
+ ],
+)
+def test_bullet_result_roles_are_exact_for_every_metric_shape(
+ metric: dict[str, object], field: str
+) -> None:
+ form_data = map_bullet_config(BulletChartConfig(metric=metric))
+ model = resolve_bullet_render_model([{field.swapcase(): "12.5"}], form_data)
+ assert model.metric_field == field.swapcase()
+ assert model.measures == [12.5]
+
+
+@pytest.mark.parametrize(
+ "rows, message",
+ [
+ ([{"other": 123}], "missing"),
+ ([{"Revenue": "not a number"}], "non-numeric text"),
+ ([{"Revenue": math.nan}], "NaN or infinite"),
+ ([{"Revenue": math.inf}], "NaN or infinite"),
+ ([{"Revenue": 1}, {}], "row 1.*missing"),
+ ([{"REVENUE": 1, "revenue": 2}], "ambiguous"),
+ ],
+)
+def test_bullet_result_validation_rejects_malformed_rows(
+ rows: list[dict[str, object]], message: str
+) -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "amount", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ with pytest.raises(BulletOutputError, match=message):
+ resolve_bullet_render_model(rows, form_data)
+
+
+def test_bullet_result_validation_accepts_null_and_numeric_strings() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "amount", "aggregate": "SUM", "label": "Revenue"},
+ dimensions=[{"name": "Region"}],
+ )
+ )
+ model = resolve_bullet_render_model(
+ [
+ {"Region": "North", "Revenue": None},
+ {"Region": "South", "Revenue": " 4.25 "},
+ ],
+ form_data,
+ )
+ assert model.measures == [0.0, 4.25]
+
+
+@pytest.mark.parametrize(
+ ("presentation", "message"),
+ [
+ ({"ranges": "10,nope"}, r"ranges\[1\].*not numeric"),
+ ({"markers": "NaN"}, r"markers\[0\].*NaN or infinite"),
+ ],
+)
+def test_bullet_result_validation_rejects_malformed_presentation(
+ presentation: dict[str, object], message: str
+) -> None:
+ form_data = {
+ **map_bullet_config(
+ BulletChartConfig(metric={"name": "amount", "aggregate": "SUM"})
+ ),
+ **presentation,
+ }
+ with pytest.raises(BulletOutputError, match=message):
+ resolve_bullet_render_model([{"SUM(amount)": 1}], form_data)
+
+
+def test_bullet_compile_accepts_empty_ungrouped_result() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {"queries": [{"data": []}]}
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ assert result.success is True
+ assert result.row_count == 0
+
+
+def test_bullet_compile_inspects_top_level_and_query_error_envelopes() -> None:
+ form_data = map_bullet_config(
+ BulletChartConfig(metric={"name": "Revenue", "aggregate": "SUM"})
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = {
+ "status": "success",
+ "queries": [{"status": "failed", "message": "warehouse timeout"}],
+ }
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ result = _compile_chart(form_data, 7)
+ assert result.success is False
+ assert "warehouse timeout" in (result.error or "")
+
+
+_MALFORMED_QUERY_ENVELOPES: list[object] = [
+ None,
+ [],
+ {},
+ {"queries": None},
+ {"queries": []},
+ {"queries": [None]},
+ {"queries": [{}]},
+ {"queries": [{"data": None}]},
+ {"queries": [{"data": []}, {"data": "not-an-array"}]},
+ {
+ "queries": [
+ {
+ "data": [{"Revenue": 12}],
+ "colnames": ["Revenue"],
+ "coltypes": [],
+ }
+ ]
+ },
+]
+
+
+def _compile_bullet_with_result(result: object) -> Any:
+ form_data = map_bullet_config(
+ BulletChartConfig(
+ metric={"name": "Revenue", "aggregate": "SUM", "label": "Revenue"}
+ )
+ )
+ factory = MagicMock()
+ factory.create.return_value = MagicMock()
+ command = MagicMock()
+ command.run.return_value = result
+ with (
+ patch(
+ "superset.common.query_context_factory.QueryContextFactory",
+ return_value=factory,
+ ),
+ patch(
+ "superset.commands.chart.data.get_data_command.ChartDataCommand",
+ return_value=command,
+ ),
+ ):
+ return _compile_chart(form_data, 7)
+
+
+@pytest.mark.parametrize("envelope", _MALFORMED_QUERY_ENVELOPES)
+def test_bullet_compile_returns_stable_error_for_malformed_envelopes(
+ envelope: object,
+) -> None:
+ result = _compile_bullet_with_result(envelope)
+ assert result.success is False
+ assert result.error_code == "CHART_COMPILE_FAILED"
+ assert result.error_obj is not None
+ assert result.error_obj.error_type == "compile_error"
+
+
+@pytest.mark.parametrize(
+ ("data", "expected_code", "expected_type"),
+ [
+ ([1], "CHART_COMPILE_FAILED", "compile_error"),
+ (
+ [{"Revenue": 10**10000}],
+ "CHART_COMPILE_FAILED",
+ "compile_error",
+ ),
+ ],
+)
+def test_bullet_compile_returns_malformed_output_for_bad_rows(
+ data: list[object],
+ expected_code: str,
+ expected_type: str,
+) -> None:
+ result = _compile_bullet_with_result({"queries": [{"data": data}]})
+ assert result.success is False
+ assert result.error_code == expected_code
+ assert result.error_obj is not None
+ assert result.error_obj.error_type == expected_type
+
+
+def test_bullet_shared_query_builder_matches_frontend_build_query() -> None:
Review Comment:
This test is named as a frontend/backend query-parity check, but it only calls the Python query builder and asserts against fields from the same config object — it never touches the frontend's `buildQuery`. If the frontend's `buildQuery` stopped forwarding `order_by`/`row_limit`, this test would still pass, so Explore and an MCP preview could silently query different rows or ordering for the same saved chart. Could this assert against the frontend `buildQuery` output (or a shared fixture) instead of just the Python builder's own config?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5936812116
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi merged PR #43770:
URL: https://github.com/apache/superset/pull/43770
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4239560831
##########
superset/mcp_service/chart/preview_utils.py:
##########
@@ -1809,7 +1815,1060 @@ def generate_xy_vega_lite_preview(
spec["transform"] = [
{"fold": series_columns, "as": [series_field, value_field]}
]
- encoding["y"] = {"field": value_field, "type": "quantitative"}
+ # Vega-Lite stacks bar and area marks colored by a nominal field
+ # by default; follow the chart's saved ``stack`` control instead.
+ encoding["y"] = {
+ "field": value_field,
+ "type": "quantitative",
+ "stack": _xy_vega_lite_stack(form_data.get("stack")),
+ }
encoding["color"] = {"field": series_field, "type": "nominal"}
+ if (
+ spec.get("mark") == "bar"
+ and encoding["y"]["stack"] is None
+ and "x" in encoding
+ ):
+ # Unstacked bars sit side by side like Explore's grouped bars.
+ # Offset channels need a discrete band x scale, so temporal
+ # values stay date-formatted through a full-precision time unit.
+ x_encoding = encoding["x"]
+ if x_encoding.get("type") == "temporal":
+ x_encoding["timeUnit"] = "yearmonthdatehoursminutesseconds"
Review Comment:
The `yearmonthdatehoursminutesseconds` time unit bins timestamps to whole seconds, not full precision as the comment says. For an unstacked `echarts_timeseries_bar` with `groupby=["region"]` and rows at `2026-01-01T12:00:00.100` and `2026-01-01T12:00:00.900`, both land in the same x band, so same-series bars overlap and one observation is hidden, while Explore keeps them distinct. The test at `test_review_query_and_preview_regressions.py:304` pins that second-level unit. Should this use a unit that keeps milliseconds, or skip the unit and bin by the actual timestamp values?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5770049518
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5805697842
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #43770:
URL: https://github.com/apache/superset/pull/43770#issuecomment-5595565541
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(mcp): add typed Bullet chart support [superset]
Posted by "aminghadersohi (via GitHub)" <gi...@apache.org>.
aminghadersohi commented on code in PR #43770:
URL: https://github.com/apache/superset/pull/43770#discussion_r4244616118
##########
superset/mcp_service/chart/schemas.py:
##########
@@ -3985,6 +4104,487 @@ def validate_unique_column_labels(self) -> "XYChartConfig":
return self
+class BulletChartConfig(BaseChartConfig):
+ """Config for bullet charts (viz_type ``bullet``)."""
+
+ # Semantic field names are exposed to MCP clients; validation aliases and the
+ # native adapter accept saved Explore ``form_data`` without weakening the
+ # unknown-field checks that catch misspelled controls.
+ model_config = ConfigDict(extra="ignore", populate_by_name=True)
+
+ # Saved query roles are internal sort context, never typed authoring input.
+ # Keeping them out of dimensions preserves omission and native SQL shapes.
+ _inherited_groupby: list[str | dict[str, Any]] | None = PrivateAttr(default=None)
+
+ @property
+ def order_dimensions(self) -> Sequence[ColumnRef | str | dict[str, Any]]:
+ """Return authored dimensions or the inherited native sort hierarchy."""
+ if self.dimensions is None and self._inherited_groupby is not None:
+ return self._inherited_groupby
+ return self.dimensions or []
+
+ chart_type: Literal["bullet"] = "bullet"
+ metric: ColumnRef = Field(
+ ...,
+ description="Numeric measure shown by each bullet bar",
+ )
+ dimensions: List[ColumnRef] | None = Field(
+ None,
+ validation_alias=AliasChoices("dimensions", "groupby"),
+ description=(
+ "Category hierarchy; one bullet row per unique combination. Omit to "
+ "keep the saved hierarchy on update; [] clears it."
+ ),
+ max_length=20,
+ )
+ filters: List[FilterConfig] | None = Field(None, max_length=100)
+ time_range: str | None = Field(
+ None,
+ min_length=1,
+ max_length=1000,
+ description=(
+ "Superset time range, e.g. 'Last 30 days' or '2025-01-01 : 2025-12-31'"
+ ),
+ )
+ row_limit: int = Field(
+ 10000,
+ ge=1,
+ le=50000,
+ description="Maximum bullet rows",
+ )
+ order_by: List[SortByConfig] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("order_by", "orderby", "order_by_cols"),
+ max_length=20,
+ description="Row order by a dimension name or the metric output label/name",
+ )
+
+ # Presentation fields map one-for-one onto Bullet/transformProps.ts controls.
+ ranges: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Qualitative range thresholds shaded behind the measure",
+ )
+ range_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("range_labels", "rangeLabels"),
+ max_length=100,
+ )
+ markers: List[float] = Field(
+ default_factory=list,
+ max_length=100,
+ description="Target values drawn as point markers",
+ )
+ marker_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_labels", "markerLabels"),
+ max_length=100,
+ )
+ marker_lines: List[float] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_lines", "markerLines"),
+ max_length=100,
+ description="Reference values drawn as vertical lines",
+ )
+ marker_line_labels: List[str] = Field(
+ default_factory=list,
+ validation_alias=AliasChoices("marker_line_labels", "markerLineLabels"),
+ max_length=100,
+ )
+ y_axis_format: str = Field(
+ "SMART_NUMBER",
+ validation_alias=AliasChoices("y_axis_format", "yAxisFormat"),
+ max_length=100,
+ )
+ show_labels: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_labels", "showLabels"),
+ )
+ show_legend: bool = Field(
+ False,
+ validation_alias=AliasChoices("show_legend", "showLegend"),
+ )
+
+ @staticmethod
+ def _adapt_native_metric(value: Any) -> Any:
+ """Translate QueryFormMetric shapes into the shared ColumnRef contract."""
+ if isinstance(value, str):
+ return {"name": value, "saved_metric": True}
+ if not isinstance(value, dict):
+ return value
+ if "expressionType" not in value:
+ # QueryObject's documented legacy saved-metric representation is a
+ # label-only object. Keep this adapter deliberately narrow: objects
+ # carrying ad-hoc fields must declare expressionType explicitly, and
+ # semantic ColumnRef objects continue through normal validation.
+ if set(value) == {"label"}:
+ label = value["label"]
+ if not isinstance(label, str) or not label or len(label) > 255:
+ raise ValueError(
+ "legacy saved metric label must be a non-empty string of "
+ "at most 255 characters"
+ )
+ return {"name": label, "saved_metric": True}
+ return value
+ expression_type = value.get("expressionType")
+ if expression_type == "SQL":
+ return {
+ "sql_expression": value.get("sqlExpression"),
+ "label": value.get("label") or value.get("sqlExpression"),
+ }
+ if expression_type != "SIMPLE":
+ raise ValueError("metric.expressionType must be 'SIMPLE' or 'SQL'")
+ column = value.get("column")
+ if isinstance(column, dict):
+ name = column.get("column_name")
+ else:
+ name = column
+ return {
+ "name": name,
+ "aggregate": value.get("aggregate"),
+ "label": value.get("label"),
+ }
+
+ @staticmethod
+ def _canonical_dimension_alias(value: Any, field_name: str) -> list[str]:
+ """Canonicalize semantic/native dimension aliases for conflict checks."""
+ if not isinstance(value, list):
+ raise ValueError(f"{field_name} must be an array")
+ canonical: list[str] = []
+ for index, item in enumerate(value):
+ name: str | None
+ if isinstance(item, str):
+ name = item
+ elif isinstance(item, ColumnRef):
+ name = item.name
+ elif isinstance(item, dict):
+ name = next(
+ (
+ item[key]
+ for key in ("name", "column_name", "column")
+ if isinstance(item.get(key), str)
+ ),
+ None,
+ )
+ else:
+ name = None
+ if not name:
+ raise ValueError(
+ f"{field_name}[{index}] must identify a physical column"
+ )
+ canonical.append(name)
+ return canonical
+
+ @staticmethod
+ def _adapt_native_order_by(value: Any) -> Any: # noqa: C901
+ if value is None:
+ return []
+ if not isinstance(value, list):
+ raise ValueError("order_by must be an array")
+ result: list[Any] = []
+ for index, entry in enumerate(value):
+ if isinstance(entry, str):
+ if len(entry) > 2000:
+ raise ValueError(f"order_by[{index}] is too long")
+ try:
+ entry = json.loads(entry)
+ except json.JSONDecodeError:
+ # A bare output/column name is the ergonomic typed form.
+ result.append({"column": entry, "ascending": False})
+ continue
+ if isinstance(entry, dict):
+ result.append(entry)
+ continue
+ if not isinstance(entry, (list, tuple)) or len(entry) != 2:
+ raise ValueError(
+ f"order_by[{index}] must be [column, ascending_boolean]"
+ )
+ target, ascending = entry
+ if isinstance(target, dict):
+ target = target.get("label") or target.get("metric_name")
Review Comment:
Fixed in b2d45096c1ae97335762b4a37fcdc25ac1ca1dfb. The native metric adapter and metric sort adapter now share output-label derivation, including the sqlExpression fallback. Regression coverage for unlabeled, empty-label, and explicitly labeled SQL metrics covers both orderby and JSON-encoded order_by_cols (four cases failed before the fix; all six pass after). Full chart unit suite: 5,640 passed, 11 skipped; touched Bullet test file: 397 passed; pre-commit passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org