You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Lionel PERRIN <li...@hotmail.com> on 2016/05/10 14:37:09 UTC
Init/Setup worker
Hello,
I’m looking for a solution to use jruby on top of spark. The only tricky point is that I
need that every worker thread has a ruby interpreter initialized.
Basically, I need to register a function to be called when each worker thread
is created : a thread local variable must be set for the ruby interpreter so
that ruby object can be deserialized.
Is there any solution to setup the worker threads before any
spark call is made using this thread ?
Regards,
Lionel
Re: Init/Setup worker
Posted by Natu Lauchande <nl...@gmail.com>.
Hi,
Not sure if this might be helpful to you :
https://github.com/ondra-m/ruby-spark .
Regards,
Natu
On Tue, May 10, 2016 at 4:37 PM, Lionel PERRIN <li...@hotmail.com>
wrote:
> Hello,
>
>
>
> I’m looking for a solution to use jruby on top of spark. The only tricky
> point is that I need that every worker thread has a ruby interpreter
> initialized. *Basically, I need to register a function to be called when
> each worker thread is created* : a thread local variable must be set for
> the ruby interpreter so that ruby object can be deserialized.
>
>
>
> Is there any solution to setup the worker threads before any spark call is
> made using this thread ?
>
>
> Regards,
>
>
> Lionel
>