You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "betodealmeida (via GitHub)" <gi...@apache.org> on 2025/03/17 16:14:14 UTC
[PR] WIP [superset]
betodealmeida opened a new pull request, #32701:
URL: https://github.com/apache/superset/pull/32701
<!---
Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/
Example:
fix(dashboard): load charts correctly
-->
### SUMMARY
<!--- Describe the change below, including rationale and design decisions -->
When filtering boolean columns in the chart builder we're using the operator `==` for `IS TRUE` and `IS FALSE`, instead of the native operators:
```typescript
[Operators.IsTrue]: { display: t('Is true'), operation: '==' },
[Operators.IsFalse]: { display: t('Is false'), operation: '==' },
```
There are two problems with this approach. First, the rendered filtered looks ugly, showing `VALUE = true` instead of `VALUE IS TRUE`:
<img width="386" alt="Screenshot 2025-03-17 at 10 44 33 AM" src="https://github.com/user-attachments/assets/ea704e1e-abee-4694-90da-2591061e72ef" />
<img width="322" alt="Screenshot 2025-03-17 at 10 44 43 AM" src="https://github.com/user-attachments/assets/32b12173-3aae-480e-9cf9-3301688b9672" />
Second, the filter in the request payload looks like this:
```json
{col: "value", op: "==", val: true}
```
When the SQLAlchemy query is generated, the filter will be built like this:
```python
where_clause_and.append(sqla_col == eq)
```
Which is iffy and error prone, specially because many databases don't support native boolean types, using integers for them. This could potentially result in broken comparisons between a boolean and an integer, for example.
It's much better to map the `IS TRUE` and `IS FALSE` to the `IS TRUE` and `IS FALSE` operators, since then the SQLAlchemy filter is generated like this:
```python
where_clause_and.append(sqla_col.is_(True))
```
This PR fixes the problem by mapping `IS TRUE` and `IS FALSE` to the correct operators (and cleaning up some code). With these changes, the UI looks correct:
<img width="319" alt="Screenshot 2025-03-17 at 11 11 26 AM" src="https://github.com/user-attachments/assets/a78e86ae-3f3b-44ca-996a-24848e2a9565" />
And the request payload:
```json
{col: "value", op: "IS TRUE"}
```
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
See discussion above.
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: boolean filters in Explore [superset]
Posted by "korbit-ai[bot] (via GitHub)" <gi...@apache.org>.
korbit-ai[bot] commented on code in PR #32701:
URL: https://github.com/apache/superset/pull/32701#discussion_r1999191908
##########
superset-frontend/src/explore/exploreUtils/index.js:
##########
@@ -300,6 +301,10 @@ export const getSimpleSQLExpression = (subject, operator, comparator) => {
[...MULTI_OPERATORS]
.map(op => OPERATOR_ENUM_TO_OPERATOR_TYPE[op].operation)
.indexOf(operator) >= 0;
+ const showComparator =
+ DISABLE_INPUT_OPERATORS.map(
+ op => OPERATOR_ENUM_TO_OPERATOR_TYPE[op].operation,
+ ).indexOf(operator) === -1;
Review Comment:
### Missing null check in operator mapping <sub></sub>
<details>
<summary>Tell me more</summary>
###### What is the issue?
The logic fails to handle edge cases where OPERATOR_ENUM_TO_OPERATOR_TYPE[op] could be undefined, potentially causing runtime errors.
###### Why this matters
If DISABLE_INPUT_OPERATORS contains an operator that doesn't exist in OPERATOR_ENUM_TO_OPERATOR_TYPE, accessing .operation on undefined will cause the application to crash.
###### Suggested change ∙ *Feature Preview*
Add a null check using optional chaining to safely handle undefined operators:
```javascript
const showComparator =
DISABLE_INPUT_OPERATORS.map(
op => OPERATOR_ENUM_TO_OPERATOR_TYPE[op]?.operation,
).indexOf(operator) === -1;
```
</details>
<sub>
[](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/84a9f738-3d10-4d22-91d6-ca21be790ffe?suggestedFixEnabled=true)
💬 Looking for more details? Reply to this comment to chat with Korbit.
</sub>
<!--- korbi internal id:73f5303a-5157-4d71-95f7-e2019cb5d4b8 -->
##########
superset-frontend/src/explore/components/controls/FilterControl/AdhocFilter/index.js:
##########
@@ -36,18 +36,8 @@ export default class AdhocFilter {
this.operator = adhocFilter.operator?.toUpperCase();
this.operatorId = adhocFilter.operatorId;
this.comparator = adhocFilter.comparator;
- if (
- [Operators.IsTrue, Operators.IsFalse].indexOf(adhocFilter.operatorId) >=
- 0
- ) {
- this.comparator = adhocFilter.operatorId === Operators.IsTrue;
- }
- if (
- [Operators.IsNull, Operators.IsNotNull].indexOf(
- adhocFilter.operatorId,
- ) >= 0
- ) {
- this.comparator = null;
+ if (DISABLE_INPUT_OPERATORS.indexOf(adhocFilter.operatorId) >= 0) {
+ this.comparator = undefined;
}
Review Comment:
### Boolean Filter Value Loss <sub></sub>
<details>
<summary>Tell me more</summary>
###### What is the issue?
Setting comparator to 'undefined' for disabled input operators when the value should be maintained for boolean operators (IsTrue/IsFalse)
###### Why this matters
This change breaks the boolean filter functionality as it loses the true/false value needed for IS TRUE/IS FALSE SQL operators, leading to incorrect query generation
###### Suggested change ∙ *Feature Preview*
```javascript
if (DISABLE_INPUT_OPERATORS.indexOf(adhocFilter.operatorId) >= 0) {
if (adhocFilter.operatorId === Operators.IsTrue) {
this.comparator = true;
} else if (adhocFilter.operatorId === Operators.IsFalse) {
this.comparator = false;
} else {
this.comparator = undefined;
}
}
```
</details>
<sub>
[](https://app.korbit.ai/feedback/aa91ff46-6083-4491-9416-b83dd1994b51/5a3e991f-289d-4bf5-b2ae-463f62fd9798?suggestedFixEnabled=true)
💬 Looking for more details? Reply to this comment to chat with Korbit.
</sub>
<!--- korbi internal id:a08fbc82-a2d5-478f-ac90-22e6e9a4c2be -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: boolean filters in Explore [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on code in PR #32701:
URL: https://github.com/apache/superset/pull/32701#discussion_r1999134847
##########
superset-frontend/src/explore/constants.ts:
##########
@@ -83,8 +83,8 @@ export const OPERATOR_ENUM_TO_OPERATOR_TYPE: {
display: t('use latest_partition template'),
operation: 'LATEST PARTITION',
},
- [Operators.IsTrue]: { display: t('Is true'), operation: '==' },
- [Operators.IsFalse]: { display: t('Is false'), operation: '==' },
+ [Operators.IsTrue]: { display: t('Is true'), operation: 'IS TRUE' },
+ [Operators.IsFalse]: { display: t('Is false'), operation: 'IS FALSE' },
Review Comment:
This is the main change in the PR, everything else is fixing the code to work properly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: boolean filters in Explore [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on code in PR #32701:
URL: https://github.com/apache/superset/pull/32701#discussion_r1999141815
##########
superset/models/helpers.py:
##########
@@ -722,6 +722,8 @@ class ExploreMixin: # pylint: disable=too-many-public-methods
}
fetch_values_predicate = None
+ normalize_columns = False
Review Comment:
I was testing this PR by building charts from a query, instead of a dataset, and I hit this error:
```
superset_app | File "/app/superset/datasource/api.py", line 118, in get_column_values
superset_app | denormalize_column = not datasource.normalize_columns
superset_app | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
superset_app | AttributeError: 'Query' object has no attribute 'normalize_columns'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: boolean filters in Explore [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on code in PR #32701:
URL: https://github.com/apache/superset/pull/32701#discussion_r1999194241
##########
superset-frontend/src/explore/components/controls/FilterControl/AdhocFilter/index.js:
##########
@@ -36,18 +36,8 @@ export default class AdhocFilter {
this.operator = adhocFilter.operator?.toUpperCase();
this.operatorId = adhocFilter.operatorId;
this.comparator = adhocFilter.comparator;
- if (
- [Operators.IsTrue, Operators.IsFalse].indexOf(adhocFilter.operatorId) >=
- 0
- ) {
- this.comparator = adhocFilter.operatorId === Operators.IsTrue;
- }
- if (
- [Operators.IsNull, Operators.IsNotNull].indexOf(
- adhocFilter.operatorId,
- ) >= 0
- ) {
- this.comparator = null;
+ if (DISABLE_INPUT_OPERATORS.indexOf(adhocFilter.operatorId) >= 0) {
+ this.comparator = undefined;
}
Review Comment:
That's the whole point of this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] WIP [superset]
Posted by "korbit-ai[bot] (via GitHub)" <gi...@apache.org>.
korbit-ai[bot] commented on PR #32701:
URL: https://github.com/apache/superset/pull/32701#issuecomment-2730101271
Based on your review schedule, I'll hold off on reviewing this PR until it's marked as ready for review. If you'd like me to take a look now, comment `/korbit-review`.
> Your admin can change your review schedule in the [Korbit Console](https://app.korbit.ai/settings)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: boolean filters in Explore [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida commented on code in PR #32701:
URL: https://github.com/apache/superset/pull/32701#discussion_r1999133632
##########
superset-frontend/src/explore/components/controls/FilterControl/AdhocFilter/index.js:
##########
@@ -103,34 +93,30 @@ export default class AdhocFilter {
}
isValid() {
- const nullCheckOperators = [Operators.IsNotNull, Operators.IsNull].map(
- op => OPERATOR_ENUM_TO_OPERATOR_TYPE[op].operation,
- );
- const truthCheckOperators = [Operators.IsTrue, Operators.IsFalse].map(
- op => OPERATOR_ENUM_TO_OPERATOR_TYPE[op].operation,
- );
if (this.expressionType === ExpressionTypes.Simple) {
- if (nullCheckOperators.indexOf(this.operator) >= 0) {
- return !!(this.operator && this.subject);
- }
- if (truthCheckOperators.indexOf(this.operator) >= 0) {
- return !!(this.subject && this.comparator !== null);
+ // operators where the comparator is not used
+ if (
+ DISABLE_INPUT_OPERATORS.map(
+ op => OPERATOR_ENUM_TO_OPERATOR_TYPE[op].operation,
+ ).indexOf(this.operator) >= 0
+ ) {
+ return this.subject;
}
+
if (this.operator && this.subject && this.clause) {
if (Array.isArray(this.comparator)) {
- if (this.comparator.length > 0) {
- // A non-empty array of values ('IN' or 'NOT IN' clauses)
- return true;
- }
Review Comment:
If we don't return `true` here we always return `false` at the end of function, so we can simplify this to a single line. Ditto below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: boolean filters in Explore [superset]
Posted by "korbit-ai[bot] (via GitHub)" <gi...@apache.org>.
korbit-ai[bot] commented on code in PR #32701:
URL: https://github.com/apache/superset/pull/32701#discussion_r1999195148
##########
superset-frontend/src/explore/components/controls/FilterControl/AdhocFilter/index.js:
##########
@@ -36,18 +36,8 @@ export default class AdhocFilter {
this.operator = adhocFilter.operator?.toUpperCase();
this.operatorId = adhocFilter.operatorId;
this.comparator = adhocFilter.comparator;
- if (
- [Operators.IsTrue, Operators.IsFalse].indexOf(adhocFilter.operatorId) >=
- 0
- ) {
- this.comparator = adhocFilter.operatorId === Operators.IsTrue;
- }
- if (
- [Operators.IsNull, Operators.IsNotNull].indexOf(
- adhocFilter.operatorId,
- ) >= 0
- ) {
- this.comparator = null;
+ if (DISABLE_INPUT_OPERATORS.indexOf(adhocFilter.operatorId) >= 0) {
+ this.comparator = undefined;
}
Review Comment:
Thanks for your contribution. As you pointed out, your pull request is intended to change the functionality where the `IS TRUE` and `IS FALSE` operators map to the correct SQL operations, providing more readable SQL queries as well as preventing potential errors when different types of databases are used. I see how this improves the implementation and I am in agreement with your changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] fix: boolean filters in Explore [superset]
Posted by "betodealmeida (via GitHub)" <gi...@apache.org>.
betodealmeida merged PR #32701:
URL: https://github.com/apache/superset/pull/32701
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org