You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Lionel PERRIN <li...@hotmail.com> on 2016/05/10 14:37:09 UTC

Init/Setup worker

Hello,

 

I’m looking for a solution to use jruby on top of spark. The only tricky point is that I
need that every worker thread has a ruby interpreter initialized.
Basically, I need to register a function to be called when each worker thread
is created : a thread local variable must be set for the ruby interpreter so
that ruby object can be deserialized.

 

Is there any solution to setup the worker threads before any
spark call is made using this thread ?
Regards,
Lionel 		 	   		  

Re: Init/Setup worker

Posted by Natu Lauchande <nl...@gmail.com>.
Hi,

Not sure if this might be helpful to you :
https://github.com/ondra-m/ruby-spark .

Regards,
Natu

On Tue, May 10, 2016 at 4:37 PM, Lionel PERRIN <li...@hotmail.com>
wrote:

> Hello,
>
>
>
> I’m looking for a solution to use jruby on top of spark. The only tricky
> point is that I need that every worker thread has a ruby interpreter
> initialized. *Basically, I need to register a function to be called when
> each worker thread is created* : a thread local variable must be set for
> the ruby interpreter so that ruby object can be deserialized.
>
>
>
> Is there any solution to setup the worker threads before any spark call is
> made using this thread ?
>
>
> Regards,
>
>
> Lionel
>