You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "tjsdud594 (via GitHub)" <gi...@apache.org> on 2025/01/16 04:56:41 UTC
[I] [SIP] Proposal to Address Preview Issues with Delta Lake and Iceberg Table [superset]
tjsdud594 opened a new issue, #31881:
URL: https://github.com/apache/superset/issues/31881
*Please make sure you are familiar with the SIP process documented*
[here](https://github.com/apache/superset/issues/5602). The SIP will be numbered by a committer upon acceptance.
## [SIP] Proposal for ...<title>
### Motivation
Link: [Issue #26449](https://github.com/apache/superset/issues/26449)
I encountered a very similar error to the case described in the issue above.
The only difference is that I am working with a Delta Lake table instead of an Iceberg table.
I would like to share this bug and propose a solution and potential improvement.
#### How to reproduce the bug
1. Create a table with partitions in the Trino catalog.
2. Open SQL Lab
3. Select the catalog, schema, and table from the drop downs.
4. You will encounter the error: "trino error: line 5:7: Column 'partition' cannot be resolved"
#### Superset Environment
- Superset version: 4.1.1
- Python version: 3.10.15
- Trino version: 467
### Proposed Change
#### Solution
Referring to the previous case, I modified the trino.py module as follows:
Module Location: `superset/db_engine_specs/trino.py : line 494~end`
**_Before_**
```
@classmethod
def get_indexes(
cls,
database: Database,
inspector: Inspector,
table: Table,
) -> list[dict[str, Any]]:
"""
Get the indexes associated with the specified schema/table.
Trino dialect raises NoSuchTableError in get_indexes if table is empty.
:param database: The database to inspect
:param inspector: The SQLAlchemy inspector
:param table: The table instance to inspect
:returns: The indexes
"""
try:
return super().get_indexes(database, inspector, table)
except NoSuchTableError:
return []
```
**_After_**
```
@classmethod
def get_indexes(
cls,
database: Database,
inspector: Inspector,
table: Table,
) -> list[dict[str, Any]]:
"""
Get the indexes associated with the specified schema/table.
Trino dialect raises NoSuchTableError in get_indexes if table is empty.
:param database: The database to inspect
:param inspector: The SQLAlchemy inspector
:param table: The table instance to inspect
:returns: The indexes
"""
try:
indexes = super().get_indexes(database, inspector, table_name, schema)
# Handle iceberg / delta tables. Even for non-partitioned tables, it returns a value
cols_ignore = {"file_count", "total_size", "data"}
if len(indexes) == 1 and indexes[0].get("name") == "partition" and cols_ignore.issubset(set(indexes[0].get("column_names", []))):
return []
return indexes
except NoSuchTableError:
return []
```
This modification ensures compatibility with both Iceberg and Delta Lake tables, improving the robustness of the solution.
### New or Changed Public Interfaces
Nope
### New dependencies
Nope
### Migration Plan and Compatibility
Nope
### Rejected Alternatives
Nope
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP] Proposal to Address Preview Issues with Delta Lake and Iceberg Table [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on issue #31881:
URL: https://github.com/apache/superset/issues/31881#issuecomment-2620064342
Nice! There's no need to write a SIP for this, since it's not a breaking change, just a fix. Feel free to submit a PR and tag me as a reviewer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP] Proposal to Address Preview Issues with Delta Lake and Iceberg Table [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida closed issue #31881: [SIP] Proposal to Address Preview Issues with Delta Lake and Iceberg Table
URL: https://github.com/apache/superset/issues/31881
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org