You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Md. Rezaul Karim" <re...@insight-centre.org> on 2017/02/06 13:02:25 UTC
How to specify "verbose GC" in Spark submit?
Dear All,
Is there any way to specify verbose GC -i.e. “-verbose:gc
-XX:+PrintGCDetails -XX:+PrintGCTimeStamps” in Spark submit?
Regards,
_________________________________
*Md. Rezaul Karim*, BSc, MSc
PhD Researcher, INSIGHT Centre for Data Analytics
National University of Ireland, Galway
IDA Business Park, Dangan, Galway, Ireland
Web: http://www.reza-analytics.eu/index.html
<http://139.59.184.114/index.html>
Re: How to specify "verbose GC" in Spark submit?
Posted by "Md. Rezaul Karim" <re...@insight-centre.org>.
Thanks, Bryan. Got your point.
Regards,
_________________________________
*Md. Rezaul Karim*, BSc, MSc
PhD Researcher, INSIGHT Centre for Data Analytics
National University of Ireland, Galway
IDA Business Park, Dangan, Galway, Ireland
Web: http://www.reza-analytics.eu/index.html
<http://139.59.184.114/index.html>
On 6 February 2017 at 13:21, Bryan Jeffrey <br...@gmail.com> wrote:
> Hello.
>
> When specifying GC options for Spark you must determine where you want the
> GC options specified - on the executors or on the driver. When you submit
> your job, for the driver, specify '--driver-java-options
> "-XX:+PrintFlagsFinal -verbose:gc", etc. For the executor specify --conf
> "spark.executor.extraJavaOptions=-XX:+PrintFlagsFinal -verbose:gc", etc.
>
> Bryan Jeffrey
>
> On Mon, Feb 6, 2017 at 8:02 AM, Md. Rezaul Karim <
> rezaul.karim@insight-centre.org> wrote:
>
>> Dear All,
>>
>> Is there any way to specify verbose GC -i.e. “-verbose:gc
>> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps” in Spark submit?
>>
>>
>>
>> Regards,
>> _________________________________
>> *Md. Rezaul Karim*, BSc, MSc
>> PhD Researcher, INSIGHT Centre for Data Analytics
>> National University of Ireland, Galway
>> IDA Business Park, Dangan, Galway, Ireland
>> Web: http://www.reza-analytics.eu/index.html
>> <http://139.59.184.114/index.html>
>>
>
>
Re: How to specify "verbose GC" in Spark submit?
Posted by Bryan Jeffrey <br...@gmail.com>.
Hello.
When specifying GC options for Spark you must determine where you want the
GC options specified - on the executors or on the driver. When you submit
your job, for the driver, specify '--driver-java-options
"-XX:+PrintFlagsFinal -verbose:gc", etc. For the executor specify --conf
"spark.executor.extraJavaOptions=-XX:+PrintFlagsFinal -verbose:gc", etc.
Bryan Jeffrey
On Mon, Feb 6, 2017 at 8:02 AM, Md. Rezaul Karim <
rezaul.karim@insight-centre.org> wrote:
> Dear All,
>
> Is there any way to specify verbose GC -i.e. “-verbose:gc
> -XX:+PrintGCDetails -XX:+PrintGCTimeStamps” in Spark submit?
>
>
>
> Regards,
> _________________________________
> *Md. Rezaul Karim*, BSc, MSc
> PhD Researcher, INSIGHT Centre for Data Analytics
> National University of Ireland, Galway
> IDA Business Park, Dangan, Galway, Ireland
> Web: http://www.reza-analytics.eu/index.html
> <http://139.59.184.114/index.html>
>