You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Kane Kim <ka...@gmail.com> on 2015/01/16 20:12:57 UTC
spark java options
I want to add some java options when submitting application:
--conf "spark.executor.extraJavaOptions=-XX:+UnlockCommercialFeatures
-XX:+FlightRecorder"
But looks like it doesn't get set. Where I can add it to make it working?
Thanks.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: spark java options
Posted by Yana Kadiyska <ya...@gmail.com>.
your quotes are in the wrong place. Try --conf
spark.executor.extraJavaOptions="-XX:+UnlockCommercialFeatures
-XX:+FlightRecorder"
On Fri, Jan 16, 2015 at 2:12 PM, Kane Kim <ka...@gmail.com> wrote:
> I want to add some java options when submitting application:
> --conf "spark.executor.extraJavaOptions=-XX:+UnlockCommercialFeatures
> -XX:+FlightRecorder"
>
> But looks like it doesn't get set. Where I can add it to make it working?
>
> Thanks.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
Re: spark java options
Posted by Marcelo Vanzin <va...@cloudera.com>.
Hi Kane,
What's the complete command line you're using to submit the app? Where
to you expect these options to appear?
On Fri, Jan 16, 2015 at 11:12 AM, Kane Kim <ka...@gmail.com> wrote:
> I want to add some java options when submitting application:
> --conf "spark.executor.extraJavaOptions=-XX:+UnlockCommercialFeatures
> -XX:+FlightRecorder"
>
> But looks like it doesn't get set. Where I can add it to make it working?
>
> Thanks.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: spark java options
Posted by Marcelo Vanzin <va...@cloudera.com>.
Hi Kane,
Here's the command line you sent me privately:
./spark-1.2.0-bin-hadoop2.4/bin/spark-submit --class
SimpleApp --conf
"spark.executor.extraJavaOptions=-XX:+UnlockCommercialFeatures
-XX:+FlightRecorder" --master local simpleapp.jar ./test.log
You're running the app in "local" mode. In that mode, the executor is
run as a thread inside the same JVM as the driver, so all
executor-related options are ignored.
Try using "local-cluster[1,1,512]" as your master, that will run a
single executor in a separate process, and then you should see your
options take effect.
On Fri, Jan 16, 2015 at 11:12 AM, Kane Kim <ka...@gmail.com> wrote:
> I want to add some java options when submitting application:
> --conf "spark.executor.extraJavaOptions=-XX:+UnlockCommercialFeatures
> -XX:+FlightRecorder"
>
> But looks like it doesn't get set. Where I can add it to make it working?
>
> Thanks.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org