You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "mistercrunch (via GitHub)" <gi...@apache.org> on 2025/07/15 19:14:01 UTC
[PR] feat: removing dup logic in sqla/models.py and models/helpers.py [superset]
mistercrunch opened a new pull request, #34177:
URL: https://github.com/apache/superset/pull/34177
Remove duplicate code between SqlaTable and ExploreMixin classes
This commit eliminates 7 duplicate methods totaling ~146 lines of code that were
introduced by PR #20281 in July 2022. The duplication occurred when @hughhhh
created the `ExploreMixin` class to enable Query objects to be visualized in Explore,
but instead of properly refactoring shared functionality, he copy-pasted methods
from SqlaTable into the new mixin.
### Root Cause Analysis
The code duplication was traced to commit e5e886739460c011a885a13b873665410045a19c
where ExploreMixin was created with methods copied from SqlaTable:
- 5 methods were exact duplicates with identical implementations
- 2 methods had diverged over time with different error handling and security practices
### Method Resolution Order Impact
SqlaTable inherits from (Model, BaseDatasource, ExploreMixin), meaning:
- SqlaTable's implementations override ExploreMixin's versions
- Query class (inheriting only from ExploreMixin) uses the ExploreMixin versions
- Both sets of methods were being used, creating genuine duplication
### Methods Removed from SqlaTable
**Exact duplicates:**
- `filter_values_handler` (60 lines) - Static method for handling filter values
- `_apply_cte` (12 lines) - Static method for appending CTEs to SQL
- `make_orderby_compatible` (28 lines) - Instance method for ORDER BY compatibility
- `text` (2 lines) - Instance method for creating TextClause objects
- `get_query_str` (4 lines) - Instance method for generating query strings
**Divergent methods consolidated:**
- `_normalize_prequery_result_type` (35 lines) - Improved to use ExploreMixin's
more robust version with better error handling for missing columns
- `changed_by_name` (5 lines) - Improved to use AuditMixinNullable's more secure
version with HTML escaping instead of plain string conversion
### Technical Improvements
Beyond eliminating duplication, this refactoring improves the codebase by:
- Using safer `.get()` method instead of direct dict access for column lookups
- Implementing proper HTML escaping for user-facing data (security improvement)
- Maintaining single source of truth for shared functionality between SqlaTable and Query
### Testing
All existing unit tests pass (8/8) and critical integration tests for the
normalize_prequery_result_type functionality have been verified. The inheritance
hierarchy ensures SqlaTable now properly inherits the more robust implementations
from ExploreMixin and AuditMixinNullable.
Fixes: Code duplication introduced in PR #20281 (feat: Visualize SqlLab.Query model data in Explore)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: removing dup logic in sqla/models.py and models/helpers.py [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch commented on code in PR #34177:
URL: https://github.com/apache/superset/pull/34177#discussion_r2208568217
##########
superset/connectors/sqla/models.py:
##########
@@ -2053,6 +1894,14 @@ def load_database(self: SqlaTable) -> None:
session = inspect(self).session # pylint: disable=disallowed-name
self.database = session.query(Database).filter_by(id=self.database_id).one()
+ def get_query_str(self, query_obj: QueryObjectDict) -> str:
+ """Returns a query as a string using ExploreMixin implementation"""
+ return ExploreMixin.get_query_str(self, query_obj)
+
+ def text(self, clause: str) -> TextClause:
+ """Returns a text clause using ExploreMixin implementation"""
+ return ExploreMixin.text(self, clause)
Review Comment:
@betodealmeida these two methods are a bit of a hack, where from my understanding the inheritance scheme is non linear in some specific cases, so we need to cherry pick some methods to skip a level of inheritance.
PR still achieves the goal of de-duping logic, but doesn't fully fix the inheritance scheme and related patchwork ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: removing dup logic in sqla/models.py and models/helpers.py [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch commented on code in PR #34177:
URL: https://github.com/apache/superset/pull/34177#discussion_r2208673481
##########
superset/connectors/sqla/models.py:
##########
@@ -1643,43 +1521,6 @@ def _get_series_orderby(
)
return ob
- def _normalize_prequery_result_type(
Review Comment:
Confirmed on slack, this comment vouches for deleting this method as opposed to the other one that's slightly different.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: removing dup logic in sqla/models.py and models/helpers.py [superset]
Posted by "mistercrunch (via GitHub)" <gi...@apache.org>.
mistercrunch merged PR #34177:
URL: https://github.com/apache/superset/pull/34177
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] feat: removing dup logic in sqla/models.py and models/helpers.py [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on code in PR #34177:
URL: https://github.com/apache/superset/pull/34177#discussion_r2208612133
##########
superset/connectors/sqla/models.py:
##########
@@ -1643,43 +1521,6 @@ def _get_series_orderby(
)
return ob
- def _normalize_prequery_result_type(
Review Comment:
The parent method seems more complete, handling cases where `column` is not a dict. This is one of the major problems with code duplication, things get fixed in one place but not another. :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org