[GitHub] [spark] erikerlandson commented on a change in pull request #25024: [SPARK-27296][SQL] User Defined Aggregators that do not ser/de on each input row

[GitBox <gi...@apache.org>]

erikerlandson commented on a change in pull request #25024: [SPARK-27296][SQL] User Defined Aggregators that do not ser/de on each input row
URL: https://github.com/apache/spark/pull/25024#discussion_r357291403
 
 

 ##########
 File path: sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala
 ##########
 @@ -79,6 +80,24 @@ class UDFRegistration private[sql] (functionRegistry: FunctionRegistry) extends
     udaf
   }
 
+  /**
+   * Registers a typed [[Aggregator]] for use with untyped Data Frames
+   *
+   * @param name the name to register under
+   * @param agg the typed Aggregator
+   * @return a UserDefinedAggregator that can be used as an aggregating UDF
+   *
+   * @since 3.0.0
+   */
+  def registerAggregator[IN: TypeTag, BUF, OUT](
 
 Review comment:
   @cloud-fan @rdblue  this iteration still has a `TypeTag` implicit creeping in , here and also in `case class UserDefinedAggregator` - the primary reason is that `Aggregator` has no input encoder associated with it, except what can be inferred from `IN`.
   
   I see two ways to square that circle:
   1. make the input-encoder a parameter for `registerAggregator` and `UserDefinedAggregator` (or an alternative overloading for java users).
   1. Add an `inputEncoder` method to `Aggregator`
   
   ATM I am feeling partial to adding `inputEncoder` to `Aggregator`, since the transition to 3.0 allows for that kind of API change and it would be symmetric with the other two aggregator methods, but the other approach is also reasonable.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org