You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "rusackas (via GitHub)" <gi...@apache.org> on 2026/01/31 13:50:40 UTC
[PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
rusackas opened a new pull request, #37590:
URL: https://github.com/apache/superset/pull/37590
### SUMMARY
Add engine specs for two Apache Software Foundation database projects that Superset can connect to:
- **Apache Phoenix** — A SQL query engine for Apache HBase, providing low-latency SQL queries over HBase data. Uses the `phoenixdb` driver with `phoenix://` dialect on default port 8765.
- **Apache IoTDB** — A time series database designed for IoT data with efficient storage and query capabilities. Uses the `apache-iotdb` driver with `iotdb://` dialect on default port 6667.
Both specs include metadata for documentation generation, project logos, connection string templates, and appropriate categorization under `APACHE_PROJECTS`.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A — metadata-only changes that appear in database documentation.
### TESTING INSTRUCTIONS
1. Verify the new engine specs are discovered: `python -c "from superset.db_engine_specs.phoenix import PhoenixEngineSpec; print(PhoenixEngineSpec.engine_name)"`
2. Verify IoTDB: `python -c "from superset.db_engine_specs.iotdb import IoTDBEngineSpec; print(IoTDBEngineSpec.engine_name)"`
3. Run the metadata lint: `cd docs && yarn lint:db-metadata`
4. Check that logos exist at `docs/static/img/databases/apache-phoenix.png` and `docs/static/img/databases/apache-iotdb.svg`
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [x] Introduces new feature or API
- [ ] Removes existing feature or API
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida merged PR #37590:
URL: https://github.com/apache/superset/pull/37590
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #37590:
URL: https://github.com/apache/superset/pull/37590#issuecomment-3828588241
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | fdd2ef667dad434bfb154f612a79c7fde03b5312 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/697e08d095ded70008e7d8e0 |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-37590--superset-docs-preview.netlify.app](https://deploy-preview-37590--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-37590--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #37590:
URL: https://github.com/apache/superset/pull/37590#issuecomment-3828565941
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on code in PR #37590:
URL: https://github.com/apache/superset/pull/37590#discussion_r2749583865
##########
superset/db_engine_specs/phoenix.py:
##########
@@ -0,0 +1,90 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from datetime import datetime
+from typing import Any, Optional
+
+from sqlalchemy import types
+
+from superset.constants import TimeGrain
+from superset.db_engine_specs.base import BaseEngineSpec, DatabaseCategory
+
+
+class PhoenixEngineSpec(BaseEngineSpec): # pylint: disable=abstract-method
+ """Dialect for Apache Phoenix"""
+
+ engine = "phoenix"
+ engine_name = "Apache Phoenix"
+
+ metadata = {
+ "description": (
+ "Apache Phoenix is a relational database layer over Apache HBase, "
+ "providing low-latency SQL queries over HBase data."
+ ),
+ "logo": "apache-phoenix.png",
+ "homepage_url": "https://phoenix.apache.org/",
+ "categories": [
+ DatabaseCategory.APACHE_PROJECTS,
+ DatabaseCategory.ANALYTICAL_DATABASES,
+ DatabaseCategory.OPEN_SOURCE,
+ ],
+ "pypi_packages": ["phoenixdb"],
+ "connection_string": "phoenix://{hostname}:{port}/",
+ "default_port": 8765,
+ "notes": (
+ "Phoenix provides a SQL interface to Apache HBase. "
+ "The phoenixdb driver connects via the Phoenix Query Server "
+ "and supports a subset of SQLAlchemy."
+ ),
+ }
+
+ _time_grain_expressions = {
+ None: "{col}",
+ TimeGrain.SECOND: (
+ "CAST(DATE_TRUNC('SECOND', CAST({col} AS TIMESTAMP)) AS TIMESTAMP)"
+ ),
+ TimeGrain.MINUTE: (
+ "CAST(DATE_TRUNC('MINUTE', CAST({col} AS TIMESTAMP)) AS TIMESTAMP)"
+ ),
+ TimeGrain.HOUR: (
+ "CAST(DATE_TRUNC('HOUR', CAST({col} AS TIMESTAMP)) AS TIMESTAMP)"
+ ),
+ TimeGrain.DAY: "CAST(DATE_TRUNC('DAY', CAST({col} AS TIMESTAMP)) AS DATE)",
+ TimeGrain.WEEK: "CAST(DATE_TRUNC('WEEK', CAST({col} AS TIMESTAMP)) AS DATE)",
+ TimeGrain.MONTH: (
+ "CAST(DATE_TRUNC('MONTH', CAST({col} AS TIMESTAMP)) AS DATE)"
+ ),
+ TimeGrain.QUARTER: (
+ "CAST(DATE_TRUNC('QUARTER', CAST({col} AS TIMESTAMP)) AS DATE)"
+ ),
+ TimeGrain.YEAR: "CAST(DATE_TRUNC('YEAR', CAST({col} AS TIMESTAMP)) AS DATE)",
Review Comment:
**Suggestion:** The time grain expressions are using the PostgreSQL-specific function `DATE_TRUNC`, which is not supported by Apache Phoenix, so any query that applies a time grain will generate invalid SQL and fail at runtime; use Phoenix's `TRUNC` function instead. [logic error]
<details>
<summary><b>Severity Level:</b> Major ⚠️</summary>
```mdx
- ❌ Time-grain grouping fails for Phoenix-backed charts.
- ❌ SQL Lab queries with time grains error on Phoenix.
- ⚠️ Affects users connecting to Phoenix databases.
```
</details>
```suggestion
"CAST(TRUNC(CAST({col} AS TIMESTAMP), 'SECOND') AS TIMESTAMP)"
),
TimeGrain.MINUTE: (
"CAST(TRUNC(CAST({col} AS TIMESTAMP), 'MINUTE') AS TIMESTAMP)"
),
TimeGrain.HOUR: (
"CAST(TRUNC(CAST({col} AS TIMESTAMP), 'HOUR') AS TIMESTAMP)"
),
TimeGrain.DAY: "CAST(TRUNC(CAST({col} AS TIMESTAMP), 'DAY') AS DATE)",
TimeGrain.WEEK: "CAST(TRUNC(CAST({col} AS TIMESTAMP), 'WEEK') AS DATE)",
TimeGrain.MONTH: (
"CAST(TRUNC(CAST({col} AS TIMESTAMP), 'MONTH') AS DATE)"
),
TimeGrain.QUARTER: (
"CAST(TRUNC(CAST({col} AS TIMESTAMP), 'QUARTER') AS DATE)"
),
TimeGrain.YEAR: "CAST(TRUNC(CAST({col} AS TIMESTAMP), 'YEAR') AS DATE)",
```
<details>
<summary><b>Steps of Reproduction ✅ </b></summary>
```mdx
1. Open a Python REPL in the repo and import the Phoenix spec:
- Run: python -c "from superset.db_engine_specs.phoenix import PhoenixEngineSpec;
print(PhoenixEngineSpec.engine_name)"
- This imports the class defined at `superset/db_engine_specs/phoenix.py:26` and loads
its attributes.
2. Inspect the time-grain templates defined on the class at
`superset/db_engine_specs/phoenix.py:54-74`:
- In the REPL evaluate:
```
from superset.db_engine_specs.phoenix import PhoenixEngineSpec
from superset.constants import TimeGrain
print(PhoenixEngineSpec._time_grain_expressions[TimeGrain.MINUTE])
```
- This returns the string containing "DATE_TRUNC(...)" located in the lines shown at
`phoenix.py:56,59,63,65-73`.
3. Use the produced template to build a query that groups by time grain (the same
templates are consumed by the query generator that uses engine specs):
- Substitute a timestamp column name, e.g.
PhoenixEngineSpec._time_grain_expressions[TimeGrain.HOUR].format(col='event_ts')
- Embed the resulting expression into a SELECT/GROUP BY and execute it against a
running Apache Phoenix instance (default port 8765 from `phoenix.py:46`) via SQL Lab or
a DB client.
4. Observe the Phoenix server returns a SQL error (function or syntax error) because the
generated SQL uses DATE_TRUNC (Postgres-style) instead of Phoenix-supported syntax:
- Error arises at query execution time in SQL Lab or when the driver (phoenixdb)
forwards the SQL to Phoenix.
Notes:
- The reproduction is concrete: the DATE_TRUNC strings exist in `phoenix.py:54-74`. Any
code path that renders time-grain expressions from PhoenixEngineSpec (the Superset query
generator consuming engine specs) will produce these invalid fragments.
```
</details>
<details>
<summary><b>Prompt for AI Agent 🤖 </b></summary>
```mdx
This is a comment left during a code review.
**Path:** superset/db_engine_specs/phoenix.py
**Line:** 57:73
**Comment:**
*Logic Error: The time grain expressions are using the PostgreSQL-specific function `DATE_TRUNC`, which is not supported by Apache Phoenix, so any query that applies a time grain will generate invalid SQL and fail at runtime; use Phoenix's `TRUNC` function instead.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on code in PR #37590:
URL: https://github.com/apache/superset/pull/37590#discussion_r2749592202
##########
superset/db_engine_specs/iotdb.py:
##########
@@ -0,0 +1,57 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from superset.db_engine_specs.base import BaseEngineSpec, DatabaseCategory
+
+
+class IoTDBEngineSpec(BaseEngineSpec): # pylint: disable=abstract-method
+ """Dialect for Apache IoTDB"""
+
+ engine = "iotdb"
+ engine_name = "Apache IoTDB"
+
+ metadata = {
+ "description": (
+ "Apache IoTDB is a time series database designed for IoT data, "
+ "with efficient storage and query capabilities for massive "
+ "time series data."
+ ),
+ "logo": "apache-iotdb.svg",
+ "homepage_url": "https://iotdb.apache.org/",
+ "categories": [
+ DatabaseCategory.APACHE_PROJECTS,
+ DatabaseCategory.TIME_SERIES,
+ DatabaseCategory.OPEN_SOURCE,
+ ],
+ "pypi_packages": ["apache-iotdb"],
+ "connection_string": ("iotdb://{username}:{password}@{hostname}:{port}/"),
Review Comment:
<div>
<div id="suggestion">
<div id="issue"><b>Incorrect Connection String Format</b></div>
<div id="fix">
The connection_string includes an incorrect trailing slash, which does not align with the documented format for apache-iotdb (e.g., iotdb://user:pass@host:port). This could prevent successful connections.
</div>
<details>
<summary>
<b>Code suggestion</b>
</summary>
<blockquote>Check the AI-generated fix before applying</blockquote>
<div id="code">
````suggestion
"connection_string": ("iotdb://{username}:{password}@{hostname}:{port}"),
````
</div>
</details>
</div>
<small><i>Code Review Run #73cd5b</i></small>
</div>
---
Should Bito avoid suggestions like this for future reviews? (<a href=https://alpha.bito.ai/home/ai-agents/review-rules>Manage Rules</a>)
- [ ] Yes, avoid them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on code in PR #37590:
URL: https://github.com/apache/superset/pull/37590#discussion_r2760862173
##########
superset/db_engine_specs/iotdb.py:
##########
@@ -0,0 +1,59 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from __future__ import annotations
+
+from superset.db_engine_specs.base import BaseEngineSpec, DatabaseCategory
+
+
+class IoTDBEngineSpec(BaseEngineSpec): # pylint: disable=abstract-method
+ """Dialect for Apache IoTDB"""
+
+ engine = "iotdb"
+ engine_name = "Apache IoTDB"
+
+ metadata = {
+ "description": (
+ "Apache IoTDB is a time series database designed for IoT data, "
+ "with efficient storage and query capabilities for massive "
+ "time series data."
+ ),
+ "logo": "apache-iotdb.svg",
+ "homepage_url": "https://iotdb.apache.org/",
+ "categories": [
+ DatabaseCategory.APACHE_PROJECTS,
+ DatabaseCategory.TIME_SERIES,
+ DatabaseCategory.OPEN_SOURCE,
+ ],
+ "pypi_packages": ["apache-iotdb"],
+ "connection_string": "iotdb://{username}:{password}@{hostname}:{port}",
+ "default_port": 6667,
+ "parameters": {
+ "username": "Database username (default: root)",
+ "password": "Database password (default: root)",
+ "hostname": "IP address or hostname",
+ "port": "Default 6667",
+ },
+ "notes": (
+ "The IoTDB SQLAlchemy dialect was written to integrate with "
+ "Apache Superset. IoTDB uses a hierarchical data model, which "
+ "is reorganized into a relational model for SQL queries."
+ ),
+ }
+
+ _time_grain_expressions = {
+ None: "{col}",
Review Comment:
I checked the docs and it indeed appears to not support any kind of timestamp truncation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #37590:
URL: https://github.com/apache/superset/pull/37590#issuecomment-3828599748
**Note on `requirements/development.txt` change:** This PR bumps the `psycopg2-binary` pin from 2.9.6 → 2.9.9. This is a pre-existing mismatch on master that was latent because the `check-python-deps` CI workflow only runs when Python files are modified — and recent master commits were all non-Python changes (GitHub Action bumps, frontend deps), so the check was being skipped. Adding `.py` files in this PR triggered the actual check, exposing the stale pin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat(db_engine_specs): add Apache Phoenix and Apache IoTDB engine specs [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #37590:
URL: https://github.com/apache/superset/pull/37590#issuecomment-3828670045
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers a full AI review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org