You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Bochun Zhang <bo...@umich.edu> on 2013/10/29 21:10:48 UTC

How did spark implement serialization of functions?

In spark REPL, we can type in a function and let it run remotely. Anybody know how is it implemented? 

I’m trying to serialize a function or a class, send it to a remote server, and let the server invoke it. I don’t care much about securities. What i’m unable to do was how to get it serialized.

Thanks in advance.

--Bochun

Re: How did spark implement serialization of functions?

Posted by Evan Chan <ev...@ooyala.com>.

You might want to have a look at ClosureCleaner.scala.


On Tue, Oct 29, 2013 at 1:10 PM, Bochun Zhang <bo...@umich.edu> wrote:

> In spark REPL, we can type in a function and let it run remotely. Anybody
> know how is it implemented?
>
> I’m trying to serialize a function or a class, send it to a remote server,
> and let the server invoke it. I don’t care much about securities. What i’m
> unable to do was how to get it serialized.
>
> Thanks in advance.
>
> --Bochun
>
>


-- 
--
Evan Chan
Staff Engineer
ev@ooyala.com  |

<http://www.ooyala.com/>
<http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><http://www.twitter.com/ooyala>