Hygienic closures for scala function serialization

[Erik Erlandson <ej...@redhat.com>]

Under certain conditions, Scala will pull an entire class instance into a closure instead of just particular value members of that class, which can knee-cap Spark's serialization of functions in multiple ways.  I tripped over this last week, and wrote up the experience on my blog.  I've since noticed a couple other user descriptions of the same scenario, so maybe the post will be interesting to the community:

http://erikerlandson.github.io/blog/2015/03/31/hygienic-closures-for-scala-function-serialization/

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org