You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by EarthsonLu <Ea...@gmail.com> on 2016/07/19 08:36:28 UTC

[SparkSQL][UDAF] CatalystTypeConverters for each update?

I just find that MutableAggregationBuffer.update will convert data for every
update, which is terrible when I use something like Map, Array.

It is hard to implement a collect_set udaf, which will be O(n^2) in this
convert semantic.

Any advice?



--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/SparkSQL-UDAF-CatalystTypeConverters-for-each-update-tp18348.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org