[GitHub] [spark] erikerlandson edited a comment on issue #25024: [SPARK-27296][SQL] User Defined Aggregators that do not ser/de on each input row

[GitBox <gi...@apache.org>]

erikerlandson edited a comment on issue #25024: [SPARK-27296][SQL] User Defined Aggregators that do not ser/de on each input row
URL: https://github.com/apache/spark/pull/25024#issuecomment-538597586
 
 
   @cloud-fan I could get behind that as a unification.
   The piece I'm ambivalent about is this one: "convert the projected internal row to `IN` type object via the encoder". What is the logic for constructing an encoder that will de-code some arbitrary projected InternalRow to type `IN`?    I could imagine being able to convert some InternalRow to a tuple up front, but we don't know what `IN` is, in general.  The programmer who defines the `Aggregator` might provide one, but that would imply adding some new features to `Aggregator`, like an encoder for `IN`, and an expected input-schema (like UDAF and IDIA have).
   
   I'll look at `ScalaReflection` further to solve the input encoding issue.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org