You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "AdamGS (via GitHub)" <gi...@apache.org> on 2026/09/07 13:42:18 UTC
[I] Nullable expression over non-null columns are planned as non-null-aware-joins [datafusion]
AdamGS opened a new issue, #25024:
URL: https://github.com/apache/datafusion/issues/25024
### Describe the bug
As seen in the query below, we currently only take column nullability into consideration when planning joins instead of full expression nullability, in cases where we join on an expression that might introduce nulls.
### To Reproduce
```sql
CREATE TABLE outer_t(id VARCHAR NOT NULL) AS VALUES ('z');
CREATE TABLE inner_t(id VARCHAR NOT NULL) AS VALUES ('a');
SELECT id
FROM outer_t
WHERE (NULLIF(id, 'z') NOT IN (SELECT id FROM inner_t)) IS NULL;
```
### Expected behavior
The query above currently returns no results, but should return a single row with the value `z` (verified with DuckDB).
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Nullable expression over non-null columns are planned as non-null-aware-joins [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #25024:
URL: https://github.com/apache/datafusion/issues/25024#issuecomment-6479357676
When fixing this bug, I think we should make the query error rather than trying to make it work / give correct results. To make it work and have confidence it will be correct, I think we need a more general approach than patches. See here for details
- https://github.com/apache/datafusion/issues/5492#issuecomment-6479090126
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Nullable expression over non-null columns are planned as non-null-aware-joins [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #25024: Nullable expression over non-null columns are planned as non-null-aware-joins
URL: https://github.com/apache/datafusion/issues/25024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Nullable expression over non-null columns are planned as non-null-aware-joins [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #25024:
URL: https://github.com/apache/datafusion/issues/25024#issuecomment-6507402854
Agreed -- thank you @raghavyadav01
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Nullable expression over non-null columns are planned as non-null-aware-joins [datafusion]
Posted by "raghavyadav01 (via GitHub)" <gi...@apache.org>.
raghavyadav01 commented on issue #25024:
URL: https://github.com/apache/datafusion/issues/25024#issuecomment-6506155178
Checked on current main (14aa185dd), this returns the right row now:
```sql
CREATE TABLE outer_t(id VARCHAR NOT NULL) AS VALUES ('z');
CREATE TABLE inner_t(id VARCHAR NOT NULL) AS VALUES ('a');
SELECT id FROM outer_t WHERE (NULLIF(id, 'z') NOT IN (SELECT id FROM inner_t)) IS NULL;
-- z
```
Same as DuckDB 1.5.6. I think this can be closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Nullable expression over non-null columns are planned as non-null-aware-joins [datafusion]
Posted by "AdamGS (via GitHub)" <gi...@apache.org>.
AdamGS commented on issue #25024:
URL: https://github.com/apache/datafusion/issues/25024#issuecomment-5571553680
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org