You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/06/09 11:06:20 UTC
[GitHub] [arrow-datafusion] alamb opened a new issue, #6611: Support sliding windows in `AggregateUDF` / User Defined Window Functions
alamb opened a new issue, #6611:
URL: https://github.com/apache/arrow-datafusion/issues/6611
### Is your feature request related to a problem or challenge?
When implementing an user defined aggregator via [AggregateUDF](https://docs.rs/datafusion/latest/datafusion/physical_plan/udaf/struct.AggregateUDF.html), it is not possible to use the [`retract_batch`](https://docs.rs/datafusion/latest/datafusion/physical_plan/trait.Accumulator.html#method.retract_batch) API to incrementally update aggregate functions used as window functions
### Describe the solution you'd like
I would like to extend somehow so user defined aggregate functions can take advantage of retract_batch
Ideally:
1. Implement the code
2. Update / add an end to end example showing how to use this API -- the existing example is https://github.com/apache/arrow-datafusion/blob/main/datafusion-examples/examples/simple_udaf.rs
### Describe alternatives you've considered
_No response_
### Additional context
@stuartcarnie found this gap while working on functionality for IOx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #6611: Support sliding windows in `AggregateUDF` / User Defined Window Functions
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6611:
URL: https://github.com/apache/arrow-datafusion/issues/6611#issuecomment-1591892795
Here is a PR that builds on the one above with the code to implement the feature (based on @stuartcarnie 's changes): https://github.com/apache/arrow-datafusion/pull/6671
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #6611: Support sliding windows in `AggregateUDF` / User Defined Window Functions
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6611:
URL: https://github.com/apache/arrow-datafusion/issues/6611#issuecomment-1591001689
Here is a version @stuartcarnie came up with on a branch: https://github.com/apache/arrow-datafusion/compare/main...stuartcarnie:arrow-datafusion:sgc/sliding_acc#diff-9c622c7093c00fc56f51d4c482e338cdb36171a878faf6d058739b61e1c693daR110-R120
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #6611: Support sliding windows in `AggregateUDF` / User Defined Window Functions
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6611:
URL: https://github.com/apache/arrow-datafusion/issues/6611#issuecomment-1591612452
Here is a PR with some tests: https://github.com/apache/arrow-datafusion/pull/6669
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on issue #6611: Support sliding windows in `AggregateUDF` / User Defined Window Functions
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #6611:
URL: https://github.com/apache/arrow-datafusion/issues/6611#issuecomment-1591193405
I am starting to work on this (based on @stuartcarnie 's work)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb closed issue #6611: Support sliding windows in `AggregateUDF` / User Defined Window Functions
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #6611: Support sliding windows in `AggregateUDF` / User Defined Window Functions
URL: https://github.com/apache/arrow-datafusion/issues/6611
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org