[GitHub] [spark] cloud-fan commented on a change in pull request #25024: [SPARK-27296][SQL] User Defined Aggregators that do not ser/de on each input row

[GitBox <gi...@apache.org>]

cloud-fan commented on a change in pull request #25024: [SPARK-27296][SQL] User Defined Aggregators that do not ser/de on each input row
URL: https://github.com/apache/spark/pull/25024#discussion_r307635973
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/expressions/udaf.scala
 ##########
 @@ -165,3 +165,140 @@ abstract class MutableAggregationBuffer extends Row {
   /** Update the ith value of this buffer. */
   def update(i: Int, value: Any): Unit
 }
+
+/**
+ * The base class for implementing user-defined imperative aggregator (UDIA).
+ *
+ * @tparam A the aggregating type: implements the aggregation logic.
+ *
+ * @since 3.0.0
+ */
+@Experimental
+abstract class UserDefinedImperativeAggregator[A] extends Serializable {
 
 Review comment:
   Internally it's executed by either `SimpleTypedAggregateExpression` or `ComplexTypedAggregateExpression`.
   * `SimpleTypedAggregateExpression`: This will be picked if the agg buffer is a single primitive type. In this case, serde is very fast and it's OK to do it for each input.
   * `ComplexTypedAggregateExpression`: This only do serde once until the buffer needs to be sent out through shuffle.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org