You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/06 18:17:21 UTC

[GitHub] [spark] tanelk edited a comment on pull request #32057: [SPARK-34961][SQL] Migrate First function from DeclarativeAggregate to TypedImperativeAggregate to improve performance

tanelk edited a comment on pull request #32057:
URL: https://github.com/apache/spark/pull/32057#issuecomment-814335217


   Perhaps @cloud-fan can comment on this. Why does `ObjectHashAggregateExec` require at least on of its aggregate functions to be `TypedImperativeAggregate`?
   
   https://github.com/apache/spark/blob/4b5fc1da752ec008468ef80a5717c8beab468387/sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/ObjectHashAggregateExec.scala#L143-L149
   
   Without that check the decision tree would be much simpler:
   1) Are all agg buffers fixed with? Use `HashAggregateExec`
   2) Is `conf.useObjectHashAggregation` enabled? Use `ObjectHashAggregateExec`
   3) Use `SortAggregateExec`
   
   From #15590 it seems, that `ObjectHashAggregateExec` is allways faster.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org