Alternative to spark.executor.extraClassPath ?

[innowireless TaeYun Kim <ta...@innowireless.co.kr>]

Hi,

 

On Spark Configuration document, spark.executor.extraClassPath is regarded
as a backwards-compatibility option. It also says that users typically
should not need to set this option.

 

Now, I must add a classpath to the executor environment (as well as to the
driver in the future, but for now I'm running YARN-client mode).

It's value is '/usr/lib/hbase/lib/*'. (I'm trying to use HBase classes.)

How can I add that to the executor environment without using
spark.executor.extraClassPath?

 

BTW, spark.executor.extraClassPath 'prepends' the classpath to the CLASSPATH
environment variable instead of appending it and seems to cause a few
problem to my application. (I've investigated launch_container.sh) Is there
a way to make it 'append' rather than 'prepend'?

 

I use Spark version 1.0.0.

 

Thanks.