You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Cheng Su (Jira)" <ji...@apache.org> on 2020/08/31 05:51:00 UTC

[jira] [Commented] (SPARK-32750) Add code-gen for SortAggregateExec

    [ https://issues.apache.org/jira/browse/SPARK-32750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17187441#comment-17187441 ] 

Cheng Su commented on SPARK-32750:
----------------------------------

Just FYI - will raise a PR in next couple of weeks. Thanks.

> Add code-gen for SortAggregateExec
> ----------------------------------
>
>                 Key: SPARK-32750
>                 URL: https://issues.apache.org/jira/browse/SPARK-32750
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: Cheng Su
>            Priority: Minor
>
> We have codegen for hash aggregate (`HashAggregateExec`) for a long time, but missing codegen for sort aggregate (`SortAggregate`). Sort aggregate is still useful in terms of performance if (1). the data after aggregate still too big to fit in memory (both hash aggregate and object hash aggregate needs to spill), (2).user can disable hash aggregate and object hash aggregate by config to prefer sort aggregate if the aggregate is after e.g. sort merge join and do not need sort at all.
> Create this Jira to add codegen support for sort aggregate.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org