You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "rusackas (via GitHub)" <gi...@apache.org> on 2026/10/11 21:46:22 UTC
[PR] refactor(db_engine_specs): dedupe boolean-coercion logic for Databricks/Spark [superset]
rusackas opened a new pull request, #45202:
URL: https://github.com/apache/superset/pull/45202
### SUMMARY
bito's post-approval review on #44355 flagged that `coerce_boolean_for_sql` and `use_equality_for_boolean_filters` were defined identically three times (`DatabricksBaseEngineSpec`, `DatabricksHiveEngineSpec`, `SparkEngineSpec`) across `databricks.py` and `spark.py`. This hoists the shared logic into a `DatabricksBooleanEqualityMixin` in `hive.py` and mixes it into all three classes, so a future change to the coercion (e.g. a third keyword) can't drift across copies.
No behavior change: the mixin's `coerce_boolean_for_sql`/`use_equality_for_boolean_filters` are byte-for-byte what each class defined on its own before.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A, backend-only refactor.
### TESTING INSTRUCTIONS
- `pytest tests/unit_tests/db_engine_specs/` — 2208 passed, 10 skipped (no changes needed; existing `test_databricks.py` coverage for `coerce_boolean_for_sql`/`use_equality_for_boolean_filters` exercises all three specs through the mixin).
- `pre-commit run --files superset/db_engine_specs/databricks.py superset/db_engine_specs/spark.py superset/db_engine_specs/hive.py` — all green (mypy, ruff, pylint, engine-spec metadata validation).
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(db_engine_specs): dedupe boolean-coercion logic for Databricks/Spark [superset]
Posted by "sadpandajoe (via GitHub)" <gi...@apache.org>.
sadpandajoe commented on code in PR #45202:
URL: https://github.com/apache/superset/pull/45202#discussion_r4254954114
##########
superset/db_engine_specs/spark.py:
##########
@@ -16,14 +16,11 @@
# under the License.
from __future__ import annotations
-from typing import Any
-
-from sqlalchemy import false, true
from sqlalchemy.dialects import registry
from superset.constants import TimeGrain
from superset.db_engine_specs.base import DatabaseCategory
-from superset.db_engine_specs.hive import HiveEngineSpec
+from superset.db_engine_specs.hive import SparkSqlBooleanEqualityMixin, HiveEngineSpec
Review Comment:
Same import-order failure here: `HiveEngineSpec` should come before `SparkSqlBooleanEqualityMixin`, otherwise ruff I001 keeps `pre-commit (current)` failing.
```suggestion
from superset.db_engine_specs.hive import HiveEngineSpec, SparkSqlBooleanEqualityMixin
```
##########
superset/db_engine_specs/databricks.py:
##########
@@ -40,7 +40,7 @@
BasicParametersMixin,
DatabaseCategory,
)
-from superset.db_engine_specs.hive import HiveEngineSpec
+from superset.db_engine_specs.hive import SparkSqlBooleanEqualityMixin, HiveEngineSpec
Review Comment:
After the rename, `SparkSqlBooleanEqualityMixin` sorts after `HiveEngineSpec`, so ruff's import sorting (I001) rejects this line and the `pre-commit (current)` job is red on this head. `spark.py` has the same import order. Could you reorder the names?
```suggestion
from superset.db_engine_specs.hive import HiveEngineSpec, SparkSqlBooleanEqualityMixin
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(db_engine_specs): dedupe boolean-coercion logic for Databricks/Spark [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #45202:
URL: https://github.com/apache/superset/pull/45202#issuecomment-6138452218
@aminghadersohi good call, renamed the mixin to `SparkSqlBooleanEqualityMixin` since Spark uses it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(db_engine_specs): dedupe boolean-coercion logic for Databricks/Spark [superset]
Posted by "codeant-ai-for-open-source[bot] (via GitHub)" <gi...@apache.org>.
codeant-ai-for-open-source[bot] commented on PR #45202:
URL: https://github.com/apache/superset/pull/45202#issuecomment-6114201745
## CodeAnt PR Risk: Low Risk
- The PR appears safe to merge; it consolidates existing boolean-filter coercion in a shared mixin for Databricks and Spark engine specs.
**Assessed commit:** `75348c73c6f2`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(db_engine_specs): dedupe boolean-coercion logic for Databricks/Spark [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #45202:
URL: https://github.com/apache/superset/pull/45202#issuecomment-6138474064
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 29b9bbc6912f27bc2585b24e14068e1b2dc32e44 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6acdb7e89f823cf09bdaeb6e |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-45202--superset-docs-preview.netlify.app](https://deploy-preview-45202--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#DctLCoMwEADQu8y6wWLBam_gMeYXDEkzYXRREe_e7N7mXdDcvu2AD8ADyLHy1h3TbxA8kDxx3sPe0HMgs6JYA5s6J6tB_OxJtBU7V-ltQhZ667zEeXxxfC4kqDQp3H8) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-45202--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(db_engine_specs): dedupe boolean-coercion logic for Databricks/Spark [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on code in PR #45202:
URL: https://github.com/apache/superset/pull/45202#discussion_r4257060419
##########
superset/db_engine_specs/databricks.py:
##########
@@ -40,7 +40,7 @@
BasicParametersMixin,
DatabaseCategory,
)
-from superset.db_engine_specs.hive import HiveEngineSpec
+from superset.db_engine_specs.hive import SparkSqlBooleanEqualityMixin, HiveEngineSpec
Review Comment:
Good catch, already reordered this in the latest push.
##########
superset/db_engine_specs/spark.py:
##########
@@ -16,14 +16,11 @@
# under the License.
from __future__ import annotations
-from typing import Any
-
-from sqlalchemy import false, true
from sqlalchemy.dialects import registry
from superset.constants import TimeGrain
from superset.db_engine_specs.base import DatabaseCategory
-from superset.db_engine_specs.hive import HiveEngineSpec
+from superset.db_engine_specs.hive import SparkSqlBooleanEqualityMixin, HiveEngineSpec
Review Comment:
Same here, sorted in the latest push.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(db_engine_specs): dedupe boolean-coercion logic for Databricks/Spark [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas merged PR #45202:
URL: https://github.com/apache/superset/pull/45202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(db_engine_specs): dedupe boolean-coercion logic for Databricks/Spark [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #45202:
URL: https://github.com/apache/superset/pull/45202#issuecomment-6114567862
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/45202?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `78.57143%` with `3 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 65.19%. Comparing base ([`90dc7ac`](https://app.codecov.io/gh/apache/superset/commit/90dc7ac32899b50ac60ea0ec7489ae5a3059d43c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`75348c7`](https://app.codecov.io/gh/apache/superset/commit/75348c73c6f21d0f226ad009f64ae05fda261193?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 2 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/superset/pull/45202?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [superset/db\_engine\_specs/hive.py](https://app.codecov.io/gh/apache/superset/pull/45202?src=pr&el=tree&filepath=superset%2Fdb_engine_specs%2Fhive.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | 62.50% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/superset/pull/45202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (90dc7ac) and HEAD (75348c7). Click for more details.
>
> <details><summary>HEAD has 4 uploads less than BASE</summary>
>
>| Flag | BASE (90dc7ac) | HEAD (75348c7) |
>|------|------|------|
>|python|7|3|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #45202 +/- ##
===========================================
- Coverage 82.96% 65.19% -17.78%
===========================================
Files 3022 3022
Lines 193903 193890 -13
Branches 45137 45135 -2
===========================================
- Hits 160870 126398 -34472
- Misses 29959 64669 +34710
+ Partials 3074 2823 -251
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/45202/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/45202/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `35.20% <78.57%> (-0.01%)` | :arrow_down: |
| [mysql](https://app.codecov.io/gh/apache/superset/pull/45202/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [postgres](https://app.codecov.io/gh/apache/superset/pull/45202/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/45202/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.02% <78.57%> (-0.01%)` | :arrow_down: |
| [python](https://app.codecov.io/gh/apache/superset/pull/45202/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.46% <78.57%> (-33.20%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/45202/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.35% <78.57%> (-0.01%)` | :arrow_down: |
| [unit](https://app.codecov.io/gh/apache/superset/pull/45202/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/45202?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org