You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Seref Arikan <se...@gmail.com> on 2014/07/09 18:09:57 UTC

Mechanics of passing functions to Spark?

Greetings,
The documentation at
http://spark.apache.org/docs/latest/programming-guide.html#passing-functions-to-spark
says:
"Note that while it is also possible to pass a reference to a method in a
class instance (as opposed to a singleton object), this requires sending
the object that contains that class along with the method"

First, could someone clarify what is meant by sending the object here? How
is the object sent to (presumably) nodes of the cluster? Is it a one time
operation per node? Why does this sound like (according to doc) a less
preferred option compared to a singleton's function? Would not the nodes
require the singleton object anyway?

Some clarification would really help.

Regards
Seref

ps: the expression "the object that contains that class" sounds a bit
unusual, is the intended meaning "the object that is the instance  of that
class" ?