Sending extraJavaOptions for Spark 1.6.1 on mesos 0.28.2 in cluster mode

["sagarcasual ." <sa...@gmail.com>]

Hello,
I have my Spark application running in cluster mode in CDH with
extraJavaOptions.
However when I am attempting a same application to run with apache mesos,
it does not recognize the properties below at all and code returns null
that reads them.

--conf spark.driver.extraJavaOptions=-Dsome.url=http://some-url \
--conf spark.executor.extraJavaOptions=-Dsome.url=http://some-url

I tried option specified in
http://stackoverflow.com/questions/35872093/missing-java-system-properties-when-running-spark-streaming-on-mesos-cluster?noredirect=1&lq=1

and still got no change in the result.

Any idea ho to achieve this in mesos.

-Regards
Sagar

Re: Sending extraJavaOptions for Spark 1.6.1 on mesos 0.28.2 in cluster mode

[Michael Gummelt <mg...@mesosphere.io>]

Probably this: https://issues.apache.org/jira/browse/SPARK-13258

As described in the JIRA, workaround is to use SPARK_JAVA_OPTS

On Mon, Sep 19, 2016 at 5:07 PM, sagarcasual . <sa...@gmail.com>
wrote:

> Hello,
> I have my Spark application running in cluster mode in CDH with
> extraJavaOptions.
> However when I am attempting a same application to run with apache mesos,
> it does not recognize the properties below at all and code returns null
> that reads them.
>
> --conf spark.driver.extraJavaOptions=-Dsome.url=http://some-url \
> --conf spark.executor.extraJavaOptions=-Dsome.url=http://some-url
>
> I tried option specified in http://stackoverflow.com/
> questions/35872093/missing-java-system-properties-when-
> running-spark-streaming-on-mesos-cluster?noredirect=1&lq=1
>
> and still got no change in the result.
>
> Any idea ho to achieve this in mesos.
>
> -Regards
> Sagar
>



-- 
Michael Gummelt
Software Engineer
Mesosphere