[jira] [Created] (SPARK-27207) There exists a bug with SortBasedAggregator where merge()/update() operations get invoked on the aggregate buffer without calling initialize

["Parth Gandhi (JIRA)" <ji...@apache.org>]

Parth Gandhi created SPARK-27207:
------------------------------------

             Summary: There exists a bug with SortBasedAggregator where merge()/update() operations get invoked on the aggregate buffer without calling initialize
                 Key: SPARK-27207
                 URL: https://issues.apache.org/jira/browse/SPARK-27207
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.0.0
            Reporter: Parth Gandhi


Normally, the aggregate operations that are invoked for an aggregation buffer for User Defined Aggregate Functions(UDAF) follow the order like initialize(), update(), eval() OR initialize(), merge(), eval(). However, after a certain threshold configurable by spark.sql.objectHashAggregate.sortBased.fallbackThreshold is reached, ObjectHashAggregate falls back to SortBasedAggregator which invokes the merge or update operation without calling initialize() on the aggregate buffer.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org