You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "fbengayed (via GitHub)" <gi...@apache.org> on 2026/06/10 03:18:46 UTC
[D] Agent using MCP cannot properly compute metrics [superset]
GitHub user fbengayed created a discussion: Agent using MCP cannot properly compute metrics
Hi
I am trying out an agent using superset MCP.
Somehow, whenever I ask it to create a chart with some aggregated values, it either uses an existing metric but omits the "saved_metric: True" parameter, or it writes the aggregation SQL on the fly, but always references a non existing column (for example, if I ask it to count the number of rows per category or per day, it will reference the in existing column "count").
Has anyone else had issues with this ?
GitHub link: https://github.com/apache/superset/discussions/40920
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Agent using MCP cannot properly compute metrics [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
GitHub user dosubot[bot] added a comment to the discussion: Agent using MCP cannot properly compute metrics
<!-- Answer -->
Yes, these are known limitations of the MCP server's metric handling that have been actively worked on. A few things to be aware of:
**The `saved_metric: true` omission** was addressed in [PR #38981](https://github.com/apache/superset/pull/38981) (merged March 31, 2026), which clarified the MCP schema and instructions so that LLMs properly use `{"name": "metric_name", "saved_metric": true}` when referencing existing dataset metrics [[1]](https://github.com/apache/superset/pull/38981).
**The non-existent column references** (e.g., referencing a "count" column that doesn't exist) happen because the MCP's metric schema previously only supported fixed aggregate functions (`SUM`, `COUNT`, etc.) applied to actual dataset columns. When an AI agent tries to compute something like "count of rows," it may hallucinate a column name rather than correctly specifying a `COUNT(*)` metric.
**Custom SQL metric support was added** in [PR #40448](https://github.com/apache/superset/pull/40448) (merged May 28, 2026), which introduced a `sql_expression` field allowing agents to express complex SQL adhoc metrics directly [[2]](https://github.com/apache/superset/pull/40448).
There's also a related [feature request (#40917)](https://github.com/apache/superset/issues/40917) that documents the broader gap around Custom SQL metrics and virtual dataset limitations [[3]](https://github.com/apache/superset/issues/40917).
**Suggestions:**
- Make sure you're on the latest Superset version to get the `saved_metric` and `sql_expression` fixes
- When prompting your agent, you may need to explicitly instruct it about the correct metric schema (e.g., use `saved_metric: true` for existing metrics, or use the `sql_expression` field for ad-hoc aggregations)
- If the agent is still referencing non-existent columns, it's likely an LLM prompting issue — the agent needs to be told to use dataset columns that actually exist (you can use the MCP's dataset inspection tools to list available columns/metrics first)
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=reply-with-mention&utm_term=apache%2Fsuperset).*
---
Share context across your team and agents. Try [Dosu](https://dosu.dev?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-tagline&utm_term=apache%2Fsuperset).
[](https://app.dosu.dev/response-feedback?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-feedback&utm_term=apache%2Fsuperset&message_id=1b10c381-52cf-4989-a84c-3c9e6f207c80) [](https://github.dosu.com/apache/superset?utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-learn-repo&utm_term=apache%2Fsuperset) [](https://app.dosu.dev/signup?referrer=openSource&source=github-footer&utm_source=github&utm_medium=bot-comment&utm_campaign=github-comment-footer-20260415&utm_content=knowledge-infrastructure-add-team&utm_term=apache%2Fsuperset)
GitHub link: https://github.com/apache/superset/discussions/40920#discussioncomment-17245094
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Agent using MCP cannot properly compute metrics [superset]
Posted by "allenjiji (via GitHub)" <gi...@apache.org>.
GitHub user allenjiji added a comment to the discussion: Agent using MCP cannot properly compute metrics
Yes, I’ve seen exactly this, and I don’t think it’s a prompting problem.
Disclosure: I work on 0sql, which exists because of this failure.
The agent hallucinates a "count" column because it was asked to write an aggregation over raw columns. There is no such column, so it invents one. Asking it to inspect the dataset first helps, but it still has to figure out the aggregation, grain, and joins every time. Sooner or later, it gets something wrong in a way that executes successfully but returns an incorrect result. Letting the agent write custom SQL expressions, which is the newer fix, makes this worse rather than better.
What has worked for us is to not let the model write SQL at all. The agent picks named measures and dimensions from a catalogue, for example, "order count" by "category" by "day", and sends that as a small spec. A planner over a governed model generates the SQL, with the aggregations and joins already defined correctly. If the agent names a field that doesn't exist, it gets a refusal with the nearest valid fields instead of a query containing a made-up column. Counting rows per category is simply a measure called "row count", defined once in the model, so the agent never needs to know it's COUNT(*).
For Superset, the planned SQL would go into a virtual dataset or SQL Lab rather than through the chart API's metric schema, so this isn't a drop-in fix for the MCP server. But if the goal is an agent that aggregates reliably, moving SQL generation out of the model is the fix.
Docs: https://0sql.io/
GitHub link: https://github.com/apache/superset/discussions/40920#discussioncomment-18947298
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Agent using MCP cannot properly compute metrics [superset]
Posted by "allenjiji (via GitHub)" <gi...@apache.org>.
GitHub user allenjiji added a comment to the discussion: Agent using MCP cannot properly compute metrics
The agent invents a "count" column because it is being asked to write the aggregation itself. One way around this is to let it pick field names only and have a planner generate the SQL. That is what 0sql does (disclosure: I work on it).
The agent sends:
```json
{
"projections": [
{ "field": "category" },
{ "field": "row count" }
]
}
```
And gets back:
```sql
SELECT T0.category AS "Category",
count(*) AS "Row Count"
FROM orders T0
GROUP BY T0.category
```
"Row count" is defined once in the model as count(*), so the agent never needs to know how it is implemented. If it asks for a field that doesn't exist, it gets a refusal with the nearest valid fields instead of SQL containing a made-up column.
For Superset, you would put the returned SQL into a virtual dataset or SQL Lab rather than the chart API.
Its free to use: https://0sql.io/
GitHub link: https://github.com/apache/superset/discussions/40920#discussioncomment-18947509
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [D] Agent using MCP cannot properly compute metrics [superset]
Posted by "allenjiji (via GitHub)" <gi...@apache.org>.
GitHub user allenjiji deleted a comment on the discussion: Agent using MCP cannot properly compute metrics
Yes, I’ve seen exactly this, and I don’t think it’s a prompting problem.
Disclosure: I work on 0sql, which exists because of this failure.
The agent hallucinates a "count" column because it was asked to write an aggregation over raw columns. There is no such column, so it invents one. Asking it to inspect the dataset first helps, but it still has to figure out the aggregation, grain, and joins every time. Sooner or later, it gets something wrong in a way that executes successfully but returns an incorrect result. Letting the agent write custom SQL expressions, which is the newer fix, makes this worse rather than better.
What has worked for us is to not let the model write SQL at all. The agent picks named measures and dimensions from a catalogue, for example, "order count" by "category" by "day", and sends that as a small spec. A planner over a governed model generates the SQL, with the aggregations and joins already defined correctly. If the agent names a field that doesn't exist, it gets a refusal with the nearest valid fields instead of a query containing a made-up column. Counting rows per category is simply a measure called "row count", defined once in the model, so the agent never needs to know it's COUNT(*).
For Superset, the planned SQL would go into a virtual dataset or SQL Lab rather than through the chart API's metric schema, so this isn't a drop-in fix for the MCP server. But if the goal is an agent that aggregates reliably, moving SQL generation out of the model is the fix.
Docs: https://0sql.io/
GitHub link: https://github.com/apache/superset/discussions/40920#discussioncomment-18947298
----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org