You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2019/05/18 03:20:00 UTC
[jira] [Assigned] (SPARK-27207) There exists a bug with
SortBasedAggregator where merge()/update() operations get invoked on the
aggregate buffer without calling initialize
[ https://issues.apache.org/jira/browse/SPARK-27207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun reassigned SPARK-27207:
-------------------------------------
Assignee: Parth Gandhi
> There exists a bug with SortBasedAggregator where merge()/update() operations get invoked on the aggregate buffer without calling initialize
> --------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-27207
> URL: https://issues.apache.org/jira/browse/SPARK-27207
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Parth Gandhi
> Assignee: Parth Gandhi
> Priority: Minor
>
> Normally, the aggregate operations that are invoked for an aggregation buffer for User Defined Aggregate Functions(UDAF) follow the order like initialize(), update(), eval() OR initialize(), merge(), eval(). However, after a certain threshold configurable by spark.sql.objectHashAggregate.sortBased.fallbackThreshold is reached, ObjectHashAggregate falls back to SortBasedAggregator which invokes the merge or update operation without calling initialize() on the aggregate buffer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org