You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "jacob-roldan (via GitHub)" <gi...@apache.org> on 2025/09/24 17:43:25 UTC
[I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
jacob-roldan opened a new issue, #35270:
URL: https://github.com/apache/superset/issues/35270
### Bug description
I have an github project with superset + pinot, where I configure the dashboard USA Births Names to use with Pinot:
https://github.com/jacob-roldan/issues-superset6-pinot
One error is in the chart "Girl Name Cloud PINOT"
This is the query:
```sql
SELECT "name", SUM(num)
FROM birth_names
WHERE ds1 >= '1925-09-24 00:00:00.000000' AND ds1 < '2025-09-24 17:30:50.000000' AND gender = 'girl' GROUP BY "name"
LIMIT 50000;
```
Where the field _name_ is escaped, but the result is wrong:
<img width="1179" height="109" alt="Image" src="https://github.com/user-attachments/assets/8b2769bd-4a9b-44d3-94b8-e9ff9dcf195f" />
That query executed with dBeaver is OK:
<img width="884" height="414" alt="Image" src="https://github.com/user-attachments/assets/8b1678ad-8b83-43bf-9d19-8c324a8ea7b7" />
The step to reproduce it using that git-repo:
```sh
//Start containers
docker-compose up -d
//Install driver for Pinot
./superset-config-pinot.sh
//Load data and import dashboard
./example_birthnames.sh
```
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
With version 5.0.0, there is a problem with Pinot driver. It's not possible to configure a connection to Pinot.
With version 4.1.4 all charts of USA Births Names dashboard are OK except one
<img width="1912" height="1992" alt="Image" src="https://github.com/user-attachments/assets/9a3171fd-ef85-42c6-8643-12ef9d005fba" />
### Checklist
- [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
Posted by "jacob-roldan (via GitHub)" <gi...@apache.org>.
jacob-roldan commented on issue #35270:
URL: https://github.com/apache/superset/issues/35270#issuecomment-3332441616
Commenting the entry in `superset/sql/parse.py` for pinot as @dosubot suggested is working.
<img width="1890" height="964" alt="Image" src="https://github.com/user-attachments/assets/6d98af0f-4688-4e0f-985d-d8914ac221fa" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #35270:
URL: https://github.com/apache/superset/issues/35270#issuecomment-3335121168
CC @betodealmeida
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #35270:
URL: https://github.com/apache/superset/issues/35270#issuecomment-3332452048
<!-- Answer -->
Great to hear the dialect workaround fixed the original query issue. The new "undefined column 'Girls'" error in the "Genders by State PINOT" widget is happening because Pinot (and Superset's Pinot integration) does not support column aliasing in SELECT or ORDER BY clauses—only real column names from the table schema are allowed in queries for Pinot datasources [see code reference](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/db_engine_specs/pinot.py).
To resolve this, check the chart or dataset configuration for any references to aliases like "Girls" and update them to use actual column names from your Pinot table. This is a current limitation of Pinot support in Superset, and avoiding aliases in your queries or chart definitions is necessary for now. Keep an eye on future Superset releases for improved Pinot compatibility.
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
---
<sup>How did I do? [Good](https://app.dosu.dev/response-feedback/19c939cd-9e50-44a4-a3c0-a9b23d3a4ab3?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/19c939cd-9e50-44a4-a3c0-a9b23d3a4ab3?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/19c939cd-9e50-44a4-a3c0-a9b23d3a4ab3?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/19c939cd-9e50-44a4-a3c0-a9b23d3a4ab3?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/19c939cd-9e50-44a4-a3c0-a9b23d3a4ab3?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/19c939cd-9e50-44a4-a3c0-a9b23d3a4ab3?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/19c939cd-9e50-44a4-a3c0-a9b23d3a4ab3?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=githu
b) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/35270)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on issue #35270:
URL: https://github.com/apache/superset/issues/35270#issuecomment-3348730778
Yeah, the issue is that [Pinot says](https://docs.pinot.apache.org/users/user-guide-query/querying-pinot):
> Pinot provides a SQL interface for querying, which uses the Calcite SQL parser to parse queries and the MYSQL_ANSI dialect.
But then it uses double quotes for identifiers, instead of backticks like MySQL (unless you have `ANSI_QUOTES` enabled).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on issue #35270:
URL: https://github.com/apache/superset/issues/35270#issuecomment-3348673750
Taking a look! Thanks for the repo for reproing this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on issue #35270:
URL: https://github.com/apache/superset/issues/35270#issuecomment-3348756606
Found this in the Calcite docs:
[MYSQL](https://calcite.apache.org/javadocAggregate/org/apache/calcite/config/Lex.html#MYSQL)
Lexical policy similar to MySQL.
[MYSQL_ANSI](https://calcite.apache.org/javadocAggregate/org/apache/calcite/config/Lex.html#MYSQL_ANSI)
Lexical policy similar to MySQL with ANSI_QUOTES option enabled.
So it looks like the only difference is the use of `ANSI_QUOTES`. I'll create a new dialect for `sqlglot` for Pinot, deriving the MySQL one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe closed issue #35270: [6.0.0-RC2] Superset+Pinot: Error with escape field names
URL: https://github.com/apache/superset/issues/35270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [6.0.0-RC2] Superset+Pinot: Error with escape field names [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #35270:
URL: https://github.com/apache/superset/issues/35270#issuecomment-3330019518
<!-- Answer -->
This issue is likely related to how Superset handles SQL dialects and identifier escaping for Pinot. There have been known problems where Superset generated queries using MySQL-style syntax (like backticks or incorrect quoting), which Pinot does not support. This caused charts to fail or return incorrect results, especially when using reserved words or quoted identifiers like `"name"`.
A fix for the SQL dialect issue was merged into the master branch, but if you're still seeing problems, it may be due to how identifier quoting is handled in the Pinot engine spec or the underlying SQLAlchemy dialect. Superset's Pinot integration does not explicitly strip or modify quotes from field names, so issues can arise if the dialect logic is too generic or not fully aligned with Pinot's requirements. There was also a recent fix for a related bug where string escaping logic was incorrectly shared between dialects, which could have affected Pinot as well [see details](https://github.com/apache/superset/pull/34895).
If you haven't already, try the workaround of removing the `'pinot'` entry from the SQL parser configuration (`superset/sql/parse.py`) as described in recent issues [here](https://github.com/apache/superset/issues/34856). This forces Superset to avoid using MySQL-style syntax for Pinot queries.
If the problem persists, it may help to test with other quoted columns to see if the issue is specific to `"name"` or affects all quoted identifiers. Also, check if your Superset instance includes the latest fixes from the master branch, as these address several dialect and escaping bugs for Pinot [reference](https://github.com/apache/superset/issues/34700).
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
---
<sup>How did I do? [Good](https://app.dosu.dev/response-feedback/c303433c-f97b-48a1-902f-4bd19eaceca4?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/c303433c-f97b-48a1-902f-4bd19eaceca4?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/c303433c-f97b-48a1-902f-4bd19eaceca4?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/c303433c-f97b-48a1-902f-4bd19eaceca4?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/c303433c-f97b-48a1-902f-4bd19eaceca4?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/c303433c-f97b-48a1-902f-4bd19eaceca4?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/c303433c-f97b-48a1-902f-4bd19eaceca4?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=githu
b) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/35270)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org