You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "iercan (via GitHub)" <gi...@apache.org> on 2025/12/19 11:25:32 UTC
[I] Boolean filters does not work for Databricks datasets [superset]
iercan opened a new issue, #36765:
URL: https://github.com/apache/superset/issues/36765
### Bug description
Creating a native filter with a boolean column causes the Databricks query to fail, likely due to an SQL translation issue.
Here is the filter we add.
<img width="486" height="222" alt="Image" src="https://github.com/user-attachments/assets/911801b7-e745-4929-949d-e9c85b0f6650" />
And this is the error whenever we try to set any value on this filter
```
Error: [DATATYPE_MISMATCH.DATA_DIFF_TYPES] Cannot resolve "(is_test_user IN (0))" due to data type mismatch: Input to `in` should all be the same type, but it's ["BOOLEAN", "INT"]. SQLSTATE: 42K09; line 6 pos 15
```
This is the query superset sending to databricks.
```
SELECT
...
FROM prod.video_coin_operation_extended
WHERE
is_test_user IN (0)
...
```
Databricks is not accepting integer values for boolean columns. Query should have sent is as `is_test_user=False`
Tested it on master and version 6.0.0
We worked around this issue by creating a string-type calculated column for filtering but would still appreciate a resolution.
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
_No response_
### Checklist
- [ ] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [ ] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "abolfazlmadanii (via GitHub)" <gi...@apache.org>.
abolfazlmadanii commented on issue #36765:
URL: https://github.com/apache/superset/issues/36765#issuecomment-5995751406
@rusackas Thanks for the ping. I'm not the author of #44355, so I can't rebase that branch.
@FrancescoCastaldi owns fix/databricks-boolean-filter-support and is the one who can rebase #44355 onto current master and resolve the conflicts so it can be reviewed/merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "pandvan (via GitHub)" <gi...@apache.org>.
pandvan commented on issue #36765:
URL: https://github.com/apache/superset/issues/36765#issuecomment-4857255102
@iercan thanks for the suggestion. It works but it's quite "unpractical" on our instance as we have dozen of datasets with many boolean columns for each one. In the meanwhile, waiting for a proper fix has merge to the master, we're using a patched 6.1 branch from https://github.com/mirco-furlan/superset/tree/6.1.0-fix-dbx-boolean that aims to solve this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "iercan (via GitHub)" <gi...@apache.org>.
iercan commented on issue #36765:
URL: https://github.com/apache/superset/issues/36765#issuecomment-4844225990
@pandvan you can workaround it by adding a string type calculated column like this
`case when boolean_column=false then 'false' else 'true' end`
Then use that column on your filter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "pandvan (via GitHub)" <gi...@apache.org>.
pandvan commented on issue #36765:
URL: https://github.com/apache/superset/issues/36765#issuecomment-4841432715
We're facing the same issue and this is blocking us to adopt Superset version 6.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "abolfazlmadanii (via GitHub)" <gi...@apache.org>.
abolfazlmadanii commented on issue #36765:
URL: https://github.com/apache/superset/issues/36765#issuecomment-5406937093
I'd like to pick this up.
[#36777](https://github.com/apache/superset/pull/36777) was pointed at the right symptom (`is_test_user IN (0)`), but it didn't fully fix the dialect issue: Hive/Databricks still bind Python `True`/`False` as integers, so the SQL becomes `col = 1` and you get `DATATYPE_MISMATCH.BINARY_OP_DIFF_TYPES`. That also hits `EQUALS` / `NOT_EQUALS` through `handle_comparison_filter`, not only `IN`.
Plan: Databricks engine-spec overrides that use SQLAlchemy `true()` / `false()` so the compiler emits `TRUE`/`FALSE` keywords, covering native-filter `IN` and comparison filters. Tests will assert the compiled SQL, not just the Python expression.
I'll open a PR against this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "iercan (via GitHub)" <gi...@apache.org>.
iercan commented on issue #36765:
URL: https://github.com/apache/superset/issues/36765#issuecomment-4728879726
This issue still persist on 6.1.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #36765: Boolean filters does not work for Databricks datasets
URL: https://github.com/apache/superset/issues/36765
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #36765:
URL: https://github.com/apache/superset/issues/36765#issuecomment-5965622513
@abolfazlmadanii #44355 is showing conflicts against master. Could you rebase so it can get reviewed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Boolean filters does not work for Databricks datasets [superset]
Posted by "FrancescoCastaldi (via GitHub)" <gi...@apache.org>.
FrancescoCastaldi commented on issue #36765:
URL: https://github.com/apache/superset/issues/36765#issuecomment-5998707309
@rusackas @abolfazlmadanii #44355 has been rebased onto current master with conflicts resolved. All unit tests for Databricks and Spark boolean filters are green (102 passed), and the PR is now clean and mergeable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org