You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "jayzhan211 (via GitHub)" <gi...@apache.org> on 2026/10/19 13:48:31 UTC
[I] `nth_value` and `approx_percentile_cont` silently ignore `DISTINCT` [datafusion]
jayzhan211 opened a new issue, #26497:
URL: https://github.com/apache/datafusion/issues/26497
### Describe the bug
`nth_value` and `approx_percentile_cont` return the same result with `DISTINCT` as without it. This is the same problem #26471 reports for `corr`, `covar_*` and `regr_*`, which #26484 fixes. Both functions are tagged `DistinctHandling::Unsupported`, but their accumulators never read `is_distinct`.
### To Reproduce
```sql
CREATE TABLE t(x DOUBLE) AS VALUES (1.0), (1.0), (1.0), (2.0), (3.0);
SELECT nth_value(x, 2 ORDER BY x), nth_value(DISTINCT x, 2 ORDER BY x) FROM t;
-- 1 1
SELECT approx_percentile_cont(x, 0.5), approx_percentile_cont(DISTINCT x, 0.5) FROM t;
-- 1 1
SELECT approx_percentile_cont(0.5) WITHIN GROUP (ORDER BY x),
approx_percentile_cont(DISTINCT 0.5) WITHIN GROUP (ORDER BY x) FROM t;
-- 1 1
```
### Expected behavior
The distinct values are `{1, 2, 3}`, so all three `DISTINCT` forms should return 2. If DataFusion does not support `DISTINCT` for these functions, the query should fail with an error instead of returning the non-distinct answer.
### Additional context
#26484 adds a `not_impl_err!` check to each accumulator. Another option is a single check in `AggregateExprBuilder::build`: return `not_impl_err!` when `is_distinct` is set and the function reports `DistinctHandling::Unsupported`. Both aggregates and window functions go through that builder, and it runs after the logical optimizer, so `SingleDistinctToGroupBy` can still rewrite cases like `corr(DISTINCT x, x)`. The default for `distinct_handling` is `Sensitive`, so user-defined functions that don't opt in are not affected. Future functions tagged `Unsupported` would then be covered automatically, and the error would come at planning time instead of execution time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `nth_value` and `approx_percentile_cont` silently ignore `DISTINCT` [datafusion]
Posted by "PreetiToppo (via GitHub)" <gi...@apache.org>.
PreetiToppo commented on issue #26497:
URL: https://github.com/apache/datafusion/issues/26497#issuecomment-6287332898
While @CuteChuanChuan is tackling the accumulators for #26497, I would love to implement the centralized architectural fix suggested in the context. I'll add a planning-time check inside `AggregateExprBuilder::build` that automatically returns a `not_impl_err!` if `is_distinct` is true and the function indicates `DistinctHandling::Unsupported`. I will open a PR for this refactor!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] `nth_value` and `approx_percentile_cont` silently ignore `DISTINCT` [datafusion]
Posted by "CuteChuanChuan (via GitHub)" <gi...@apache.org>.
CuteChuanChuan commented on issue #26497:
URL: https://github.com/apache/datafusion/issues/26497#issuecomment-6270060231
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org