You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "buraksenn (via GitHub)" <gi...@apache.org> on 2026/03/16 15:16:16 UTC
[PR] [Minor]: support window functions in order by expressions [datafusion]
buraksenn opened a new pull request, #20963:
URL: https://github.com/apache/datafusion/pull/20963
## Which issue does this PR close?
- Closes #608.
## Rationale for this change
#608 has details but as @alamb describes in https://github.com/apache/datafusion/issues/608#issuecomment-4034162591
```
SELECT c2
FROM test
ORDER BY max(c3) OVER (ORDER BY c9);
```
fails with:
```
This feature is not implemented: Physical plan does not support logical expression WindowFunction(WindowFunction { fun: AggregateUDF(AggregateUDF { inner: Max { signature: Signature { type_signature: UserDefined, volatility: Immutable, parameter_names: None } } }), params: WindowFunctionParams { args: [Column(Column { relation: Some(Bare { table: "test" }), name: "c3" })], partition_by: [], order_by: [Sort { expr: Column(Column { relation: Some(Bare { table: "test" }), name: "c9" }), asc: true, nulls_first: false }], window_frame: WindowFrame { units: Range, start_bound: Preceding(Int32(NULL)), end_bound: CurrentRow, is_causal: false }, filter: None, null_treatment: None, distinct: false } })
```
## What changes are included in this PR?
- relevant change
- slt and unit tests
## Are these changes tested?
Yes added both unit tests for plan and slt tests for checking output
## Are there any user-facing changes?
Additive user changes that users can now use window in order by expressions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [Minor]: support window functions in order by expressions [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #20963:
URL: https://github.com/apache/datafusion/pull/20963#issuecomment-4144398484
🎉
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [Minor]: support window functions in order by expressions [datafusion]
Posted by "jonahgao (via GitHub)" <gi...@apache.org>.
jonahgao commented on code in PR #20963:
URL: https://github.com/apache/datafusion/pull/20963#discussion_r2988447937
##########
datafusion/sql/src/select.rs:
##########
@@ -297,14 +297,23 @@ impl<S: ContextProvider> SqlToRel<'_, S> {
plan
};
- // The outer expressions we will search through for window functions.
- // Window functions may be sourced from the SELECT list or from the QUALIFY expression.
- let windows_expr_haystack = select_exprs_post_aggr
- .iter()
- .chain(qualify_expr_post_aggr.iter());
+ // The window expressions from SELECT and QUALIFY only, used to validate that
+ // QUALIFY is used with window functions (ORDER BY window functions don't count).
+ let qualify_window_func_exprs = find_window_exprs(
Review Comment:
It seems this is only useful when there is a QUALIFY clause. Can we make its execution conditional based on the presence of such a clause?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [Minor]: support window functions in order by expressions [datafusion]
Posted by "buraksenn (via GitHub)" <gi...@apache.org>.
buraksenn commented on code in PR #20963:
URL: https://github.com/apache/datafusion/pull/20963#discussion_r2984089766
##########
datafusion/sqllogictest/test_files/window_order_by.slt:
##########
@@ -0,0 +1,91 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
I was thinking since it is a new feature and intersection I can create a test file will do of course. I've moved them to `window.slt`. Thanks for the review @alamb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [Minor]: support window functions in order by expressions [datafusion]
Posted by "jonahgao (via GitHub)" <gi...@apache.org>.
jonahgao merged PR #20963:
URL: https://github.com/apache/datafusion/pull/20963
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [Minor]: support window functions in order by expressions [datafusion]
Posted by "buraksenn (via GitHub)" <gi...@apache.org>.
buraksenn commented on code in PR #20963:
URL: https://github.com/apache/datafusion/pull/20963#discussion_r2990756019
##########
datafusion/sql/src/select.rs:
##########
@@ -297,14 +297,23 @@ impl<S: ContextProvider> SqlToRel<'_, S> {
plan
};
- // The outer expressions we will search through for window functions.
- // Window functions may be sourced from the SELECT list or from the QUALIFY expression.
- let windows_expr_haystack = select_exprs_post_aggr
- .iter()
- .chain(qualify_expr_post_aggr.iter());
+ // The window expressions from SELECT and QUALIFY only, used to validate that
+ // QUALIFY is used with window functions (ORDER BY window functions don't count).
+ let qualify_window_func_exprs = find_window_exprs(
Review Comment:
I’ve taken a note and will open a followup PR if suitable. Thanks for the review @jonahgao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] [Minor]: support window functions in order by expressions [datafusion]
Posted by "buraksenn (via GitHub)" <gi...@apache.org>.
buraksenn commented on code in PR #20963:
URL: https://github.com/apache/datafusion/pull/20963#discussion_r2990756019
##########
datafusion/sql/src/select.rs:
##########
@@ -297,14 +297,23 @@ impl<S: ContextProvider> SqlToRel<'_, S> {
plan
};
- // The outer expressions we will search through for window functions.
- // Window functions may be sourced from the SELECT list or from the QUALIFY expression.
- let windows_expr_haystack = select_exprs_post_aggr
- .iter()
- .chain(qualify_expr_post_aggr.iter());
+ // The window expressions from SELECT and QUALIFY only, used to validate that
+ // QUALIFY is used with window functions (ORDER BY window functions don't count).
+ let qualify_window_func_exprs = find_window_exprs(
Review Comment:
I’ve taken a note and will open a followup PR if suitable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org