You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "lyne7-sc (via GitHub)" <gi...@apache.org> on 2026/08/31 07:47:46 UTC
[I] Aggregate dynamic filter can prune rows required by unsupported expressions [datafusion]
lyne7-sc opened a new issue, #24816:
URL: https://github.com/apache/datafusion/issues/24816
### Describe the bug
When an `AggregateExec` contains both supported aggregate expressions such as `MIN(a)` and an unsupported expression such as `MIN(c + 1)`, DataFusion builds a dynamic filter using only the supported expressions.
Because this filter is applied to the shared input scan, it can prune rows that are still required by the unsupported aggregate and produce incorrect results.
### To Reproduce
Use two Parquet files containing:
```text
file_0: (1, 12, 100), (8, 4, 70)
file_1: (1, 6, 90), (8, 12, 110)
```
Then run:
```sql
SELECT MIN(a), MAX(a), MAX(b), MIN(c + 1)
FROM t;
```
The expected result for `MIN(c + 1)` is 71.
Currently, the supported aggregates can produce the following dynamic filter:
`a < 1 OR a > 8 OR b > 12`
This filter does not account for `MIN(c + 1)`. Depending on execution order, it can prune file_0 and produce 91 instead of 71.
### Expected behavior
Aggregate dynamic filtering should only be enabled when a safe predicate can be produced for every aggregate expression.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Aggregate dynamic filter can prune rows required by unsupported expressions [datafusion]
Posted by "kosiew (via GitHub)" <gi...@apache.org>.
kosiew closed issue #24816: Aggregate dynamic filter can prune rows required by unsupported expressions
URL: https://github.com/apache/datafusion/issues/24816
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Aggregate dynamic filter can prune rows required by unsupported expressions [datafusion]
Posted by "lyne7-sc (via GitHub)" <gi...@apache.org>.
lyne7-sc commented on issue #24816:
URL: https://github.com/apache/datafusion/issues/24816#issuecomment-5475362872
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org