You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jaonary Rabarisoa <ja...@gmail.com> on 2014/08/27 13:23:29 UTC
External dependencies management with spark
Dear all,
I'm looking for an efficient way to manage external dependencies. I know
that one can add .jar or .py dependencies easily but how can I handle other
type of dependencies. Specifically, I have some data processing algorithm
implemented with other languages (ruby, octave, matlab, c++) and I need to
put all of these libraries component (set of .rb, .m, .so files) on my
workers in order to able to call them as external process.
What is the best way to achieve this ?
Cheers,
Jaonary