You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Daniel Mateus Pires <dm...@gmail.com> on 2019/06/05 20:47:15 UTC

Blog post: DataFrame.transform -- Spark function composition

Hi everyone!

I just published this blog post on how Spark Scala custom transformations
can be re-arranged to better be composed and used within .transform:


https://medium.com/@dmateusp/dataframe-transform-spark-function-composition-eb8ec296c108

I found the discussions in this group to be largely around issues /
performance, this post only concerns itself with code readability,
hopefully not off-topic!

I welcome any feedback I can get :)

Daniel