You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "hanke580 (via GitHub)" <gi...@apache.org> on 2026/09/16 09:42:39 UTC
[I] Internal error: CSE-extracted `__common_expr_1` has mismatched nullability between logical and physical plans when the expression is under an aggregate [datafusion]
hanke580 opened a new issue, #25369:
URL: https://github.com/apache/datafusion/issues/25369
### Describe the bug
An aggregate over an expression that appears twice fails to plan. Common subexpression
elimination extracts the shared expression as `__common_expr_1`, and its nullability is
computed one way for the logical plan and another for the physical plan; the aggregate is
the operator that checks the two agree, so the query never runs:
```
Internal error: Physical input schema should be the same as the one converted from logical
input schema. Differences:
- field nullability at index 0 [__common_expr_1]: (physical) true vs (logical) false.
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this
by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues
```
Reproduces on **datafusion-cli 55.1.0** (latest crates.io release) and on the
**datafusion 54.0.0** Python wheel.
### To Reproduce
Three rows are enough, and no session settings are changed:
```sql
CREATE TABLE t(a INT, b INT) AS VALUES (1,1),(2,2),(3,3);
SELECT sum(CASE WHEN (CASE WHEN (a > 1 AND b < 3) THEN a ELSE 0 END)
BETWEEN 1 AND 2 THEN 1 ELSE 0 END)
FROM t;
```
`datafusion-cli -f repro.sql` raises the Internal error above.
Three ingredients are each necessary -- removing any one makes the query plan and run
normally. Measured on 55.1.0:
| query | result |
|---|---|
| aggregate + two columns + duplicated expression (above) | **Internal error** |
| `count()` instead of `sum()` | **Internal error** |
| `BETWEEN` written out as `>= 1 AND <= 2` | **Internal error** |
| the same expression with no aggregate | planned and ran |
| one column used twice (`a > 1 AND a < 3`) instead of two | planned and ran |
| a single comparison, so nothing is duplicated | planned and ran |
| one condition in the inner `CASE` | planned and ran |
So it needs (1) an **aggregate** over the expression, (2) an inner `CASE` condition
referencing **two distinct columns** -- the same column twice is fine, which points at
nullability inference rather than at `CASE` itself -- and (3) the expression appearing
**twice**, which is what invites CSE to extract it. `BETWEEN` duplicates it implicitly;
writing the two comparisons out by hand fails the same way.
### Expected behavior
The query plans and returns a result, as the same expression does without the aggregate.
For the three-row table above the answer is 1.
### Additional context
- Reproduced on **datafusion-cli 55.1.0** (built from crates.io with `cargo install
datafusion-cli --version 55.1.0 --locked`) and on the **datafusion 54.0.0** Python wheel
(the newest on PyPI). All seven rows of the table above behave identically on both.
- It fires regardless of batch size, target partition count, and optimizer settings.
- I searched the tracker for the error text and for `__common_expr` together with
nullability and did not find this reported. The nearest items look like different
triggers: #19049 is the same message but arises from `optimize_projections` over a
UNION (there is no union here); #15190 (CSE extracting window functions) and #14540
(push-down vs CSE) are CSE defects but not nullability mismatches under an aggregate;
#17801 (a TPC-DS nullability mismatch) is closed.
- Environment: Ubuntu 22.04.2 LTS, x86_64, kernel 5.15.0-187-generic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Internal error: CSE-extracted `__common_expr_1` has mismatched nullability between logical and physical plans when the expression is under an aggregate [datafusion]
Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb closed issue #25369: Internal error: CSE-extracted `__common_expr_1` has mismatched nullability between logical and physical plans when the expression is under an aggregate
URL: https://github.com/apache/datafusion/issues/25369
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Internal error: CSE-extracted `__common_expr_1` has mismatched nullability between logical and physical plans when the expression is under an aggregate [datafusion]
Posted by "1fanwang (via GitHub)" <gi...@apache.org>.
1fanwang commented on issue #25369:
URL: https://github.com/apache/datafusion/issues/25369#issuecomment-5707145834
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org