You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "rusackas (via GitHub)" <gi...@apache.org> on 2025/08/12 19:23:11 UTC
[PR] fix(sql): Add Impala dialect support to sqlglot parser [superset]
rusackas opened a new pull request, #34662:
URL: https://github.com/apache/superset/pull/34662
## Summary
This PR fixes dataset creation for Impala databases by adding Impala dialect support to the sqlglot SQL parser.
## Motivation
- Fixes #32143
- Users with Impala databases were unable to create datasets after upgrading to Superset 4.1.1
- The issue was caused by sqlglot not recognizing the Impala SQL dialect, leading to parse errors
## Solution
Since Impala's SQL syntax is very similar to Hive, this PR maps the Impala engine to use the Hive dialect in sqlglot. This is consistent with how the Ascend engine (which is also Impala-based) is already configured.
## Testing
- Added comprehensive unit tests for Impala SQL parsing
- Tests verify that common Impala SQL constructs (SELECT, JOIN, INSERT, CREATE TABLE) are properly parsed
- All existing tests continue to pass
## Checklist
- [x] Has associated issue: #32143
- [x] Required feature flags: None
- [x] API changes: None
- [x] Original bug cause identified: Missing Impala dialect mapping in SQLGLOT_DIALECTS
- [x] Unit tests added
- [x] Pre-commit checks pass
🤖 Generated with [Claude Code](https://claude.ai/code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sql): Add Impala dialect support to sqlglot parser [superset]
Posted by "korbit-ai[bot] (via GitHub)" <gi...@apache.org>.
korbit-ai[bot] commented on code in PR #34662:
URL: https://github.com/apache/superset/pull/34662#discussion_r2270953564
##########
superset/sql/parse.py:
##########
@@ -81,7 +81,7 @@
"hana": Dialects.POSTGRES,
"hive": Dialects.HIVE,
# "ibmi": ???
- # "impala": ???
+ "impala": Dialects.HIVE,
Review Comment:
### Incomplete Impala SQL Support <sub></sub>
<details>
<summary>Tell me more</summary>
###### What is the issue?
Direct mapping of Impala to Hive dialect may not handle Impala-specific SQL syntax variations correctly.
###### Why this matters
While Impala is similar to Hive, it has some syntax variations and features that differ from Hive. Using Hive's dialect directly could lead to incorrect parsing or errors when processing Impala-specific SQL constructs.
###### Suggested change ∙ *Feature Preview*
Consider creating a custom Impala dialect that extends the Hive dialect to handle Impala-specific syntax variations:
```python
class ImpalaDialect(Dialects.HIVE):
# Override or add methods to handle Impala-specific syntax
pass
# Then update the mapping
"impala": ImpalaDialect,
```
###### Provide feedback to improve future suggestions
[](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/52c2a272-5c3e-4998-9017-3e3e0bfeb806/upvote) [](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/52c2a272-5c3e-4998-9017-3e3e0bfeb806?what_not_true=true) [](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/52c2a272-5c3e-4998-9017-3e3e0bfeb806?what_out_of_scope=true) [](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/52c2a272-5c3e-4998-9017-3e3e0bfeb806?what_not_in_standard=true) [](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/52c2a272-5c3e-4998-9017-3e3e0bfeb806)
</details>
<sub>
💬 Looking for more details? Reply to this comment to chat with Korbit.
</sub>
<!--- korbi internal id:ff31ae9c-b59a-4296-880a-23d501307304 -->
[](ff31ae9c-b59a-4296-880a-23d501307304)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sql): Add Impala dialect support to sqlglot parser [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #34662:
URL: https://github.com/apache/superset/pull/34662#discussion_r2274173393
##########
tests/unit_tests/sql/test_impala_dialect.py:
##########
@@ -0,0 +1,73 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Tests for Impala dialect support in sqlglot."""
+
+from sqlglot import Dialects
+
+from superset.sql.parse import SQLGLOT_DIALECTS, SQLScript, Table
+
+
+def test_impala_dialect_mapped() -> None:
+ """Test that Impala is correctly mapped to Hive dialect."""
+ assert "impala" in SQLGLOT_DIALECTS
Review Comment:
@sadpandajoe feel free to simplify this, if it's just re-testing Hive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sql): Add Impala dialect support to sqlglot parser [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas merged PR #34662:
URL: https://github.com/apache/superset/pull/34662
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sql): Add Impala dialect support to sqlglot parser [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #34662:
URL: https://github.com/apache/superset/pull/34662#issuecomment-3254980556
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Default Agent` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sql): Add Impala dialect support to sqlglot parser [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #34662:
URL: https://github.com/apache/superset/pull/34662#discussion_r2286666834
##########
tests/unit_tests/sql/test_impala_dialect.py:
##########
@@ -0,0 +1,73 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Tests for Impala dialect support in sqlglot."""
Review Comment:
I don't think there is a hive dialect test. Maybe just make this the hive dialect one and call it a day?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix(sql): Add Impala dialect support to sqlglot parser [superset]
Posted by "villebro (via GitHub)" <gi...@apache.org>.
villebro commented on code in PR #34662:
URL: https://github.com/apache/superset/pull/34662#discussion_r2322900378
##########
tests/unit_tests/sql/test_hive_dialect.py:
##########
@@ -0,0 +1,73 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Tests for Hive dialect support in sqlglot."""
+
+from sqlglot import Dialects
+
+from superset.sql.parse import SQLGLOT_DIALECTS, SQLScript, Table
+
+
+def test_impala_dialect_mapped() -> None:
+ """Test that Impala is correctly mapped to Hive dialect."""
+ assert "impala" in SQLGLOT_DIALECTS
+ assert SQLGLOT_DIALECTS["impala"] == Dialects.HIVE
Review Comment:
Could we maybe move this out to a `test_impala_dialect.py`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org