You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "jayzhan211 (via GitHub)" <gi...@apache.org> on 2024/10/09 00:54:42 UTC

[I] Ordered Set Aggregate Functions [datafusion]

jayzhan211 opened a new issue, #12824:
URL: https://github.com/apache/datafusion/issues/12824

   ### Is your feature request related to a problem or challenge?
   
   DataFusion doesn't support ordered-set aggregate functions yet. 
   
   Those functions are supported in [Postgres](https://www.postgresql.org/docs/9.4/functions-aggregate.html), [DuckDB](https://duckdb.org/docs/sql/functions/aggregates#ordered-set-aggregate-functions), so I think we should support them here as core functions
   
   ![Screenshot 2024-10-09 at 8 49 19 AM](https://github.com/user-attachments/assets/d249a617-1efd-4259-93cc-f23781b29a1b)
   
   Related PR, https://github.com/apache/datafusion/issues/11732.
   
   - [ ] mode() WITHIN GROUP (ORDER BY sort_expression)
   - [ ] percentile_cont(fraction) WITHIN GROUP (ORDER BY sort_expression)
   - [ ] percentile_cont(fractions) WITHIN GROUP (ORDER BY sort_expression)
   - [ ] percentile_disc(fraction) WITHIN GROUP (ORDER BY sort_expression)
   - [ ] percentile_disc(fractions) WITHIN GROUP (ORDER BY sort_expression)
   
   We have `approx_percentile_cont` and `approx_percentile_cont_with_weight` already, maybe we can start from these two functions
   
   - [ ] approx_percentile_cont(fraction) WITHIN GROUP (ORDER BY sort_expression)
   - [ ] approx_percentile_cont_with_weight(fraction) WITHIN GROUP (ORDER BY sort_expression)
   
   ### Describe the solution you'd like
   
   _No response_
   
   ### Describe alternatives you've considered
   
   _No response_
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Ordered Set Aggregate Functions [datafusion]

Posted by "Garamda (via GitHub)" <gi...@apache.org>.
Garamda commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-2401308768

   Alright. Before I take the issue, let me be clear. What I understood is that the things to be implemented are those five functions below,
   
   <img width="1066" alt="Screenshot 2024-10-09 at 1 42 05 PM" src="https://github.com/user-attachments/assets/c0f51801-66db-481a-b210-106969b50556">
   
   ---
   
   not including those two functions, is that correct?
   
   <img width="1083" alt="Screenshot 2024-10-09 at 1 25 45 PM" src="https://github.com/user-attachments/assets/87188681-071b-4f89-93b4-43941a9b64f9">
   
   Let me double check because there are 7 tasks in total in this issue checklist.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Epic: Ordered Set Aggregate Functions [datafusion]

Posted by "jonathanc-n (via GitHub)" <gi...@apache.org>.
jonathanc-n commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-2408179355

   @Garamda @jayzhan211 After the WITHIN GROUP is added, I would like to work on this for the ORDER BY impelmentations for approx_percentile_cont and approx_percentile_cont_with_weight


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Ordered Set Aggregate Functions [datafusion]

Posted by "Garamda (via GitHub)" <gi...@apache.org>.
Garamda commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-2401138148

   Hi, I'm new here. I want to take the issue or participate in developing one of those functions. Would it be possible?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Epic: Ordered Set Aggregate Functions [datafusion]

Posted by "Garamda (via GitHub)" <gi...@apache.org>.
Garamda commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-2629341277

   PR is ready to get review : https://github.com/apache/datafusion/pull/13511
   
   cc. @jayzhan211


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Epic: Ordered Set Aggregate Functions [datafusion]

Posted by "Garamda (via GitHub)" <gi...@apache.org>.
Garamda commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-2401659938

   I see. So there are 3 sub tasks, I think.
   
   1. fix the syntax of `approx_percentile_cont` and `approx_percentile_cont_with_weight` to support `WITHIN GROUP`  (#11732)
   2. fix the same functions to support `ORDER BY sort_expression`
   3. After finishing step 1 & 2, implement new functions below (since `WITHIN GROUP` & `ORDER BY` will be reused)
     - mode() WITHIN GROUP (ORDER BY sort_expression)
     - percentile_cont(fraction) WITHIN GROUP (ORDER BY sort_expression)
     - percentile_cont(fractions) WITHIN GROUP (ORDER BY sort_expression)
     - percentile_disc(fraction) WITHIN GROUP (ORDER BY sort_expression)
     - percentile_disc(fractions) WITHIN GROUP (ORDER BY sort_expression)
   
   If there is nothing that I missed, I can start from step 1.
   So I think I can make a `take` comment on #11732 and get assigned, right?
   
   (or if you want to reorganize sub tasks and make new issues for each one since it was changed into `EPIC`, I can wait.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Epic: Ordered Set Aggregate Functions [datafusion]

Posted by "Jefffrey (via GitHub)" <gi...@apache.org>.
Jefffrey commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-3413814670

   `percentile_cont(fraction)` added by #17988


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Ordered Set Aggregate Functions [datafusion]

Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-2401201975

   > Hi, I'm new here. I want to take the issue or participate in developing one of those functions. Would it be possible?
   
   Sure, just type `take` then you will be assigned


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Ordered Set Aggregate Functions [datafusion]

Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-2401603635

   I think the first step is to solve #11732, fix the syntax of `approx_percentile_cont` since there is no `WITHIN GROUP` currently


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] Epic: Ordered Set Aggregate Functions [datafusion]

Posted by "Garamda (via GitHub)" <gi...@apache.org>.
Garamda commented on issue #12824:
URL: https://github.com/apache/datafusion/issues/12824#issuecomment-2449295337

   This task has taken longer than I initially expected, because recently I haven’t been able to fully focus on it. Nevertheless, I believe I can complete it successfully.
   
   However, if this is a delay for the datafusion roadmap or has any negative effect on the ecosystem, I am open to passing this issue to a more experienced committer and shifting my focus to a other smaller issue.
   
   If there’s still room for me to continue, I would like to see this issue through to completion.
   
   I’m sharing this update to inform the current status, especially since some may be waiting for this issue’s completion.
   
   cc. @jayzhan211 @jonathanc-n 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org