You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Takeshi Yamamuro (Jira)" <ji...@apache.org> on 2020/09/01 00:54:00 UTC

[jira] [Commented] (SPARK-32750) Add code-gen for SortAggregateExec

    [ https://issues.apache.org/jira/browse/SPARK-32750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17188071#comment-17188071 ] 

Takeshi Yamamuro commented on SPARK-32750:
------------------------------------------

Yea, I don't remember the impl. details in the PR, but you might be able to refer to parts of them ;)

> Add code-gen for SortAggregateExec
> ----------------------------------
>
>                 Key: SPARK-32750
>                 URL: https://issues.apache.org/jira/browse/SPARK-32750
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: Cheng Su
>            Priority: Minor
>
> We have codegen for hash aggregate (`HashAggregateExec`) for a long time, but missing codegen for sort aggregate (`SortAggregate`). Sort aggregate is still useful in terms of performance if (1). the data after aggregate still too big to fit in memory (both hash aggregate and object hash aggregate needs to spill), (2).user can disable hash aggregate and object hash aggregate by config to prefer sort aggregate if the aggregate is after e.g. sort merge join and do not need sort at all.
> Create this Jira to add codegen support for sort aggregate.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org