You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2019/06/06 16:03:00 UTC
[jira] [Resolved] (SPARK-27704) Change default class loader to
ParallelGC
[ https://issues.apache.org/jira/browse/SPARK-27704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-27704.
-------------------------------
Resolution: Not A Problem
Do you mean garbage collector? the user can already choose the GC by setting JVM options. There is no default in Spark. I don't think there's anything to do here.
> Change default class loader to ParallelGC
> -----------------------------------------
>
> Key: SPARK-27704
> URL: https://issues.apache.org/jira/browse/SPARK-27704
> Project: Spark
> Issue Type: Sub-task
> Components: Build
> Affects Versions: 3.0.0
> Reporter: Mihaly Toth
> Priority: Major
>
> In JDK 11 the default class loader changed from ParallelGC to G1GC. Even though this gc performs better on pause times and interactivity, most of the tasks that need to be processed are more sensitive to throughput and the to the amount of memory. G1 sacrifices these to some extend to avoid the big pauses. As a result the user may perceive a regression compared to JDK 8. Even worse, the regression may not be limited to performance only but some jobs may start failing in case they do not fit into the memory they used to be happy with when running with previous JDK.
> Some other kind of apps, like streaming ones, may rather use G1 because of their more interactive, more realtime needs.
> With this jira it is proposed to have a configurable default GC for all spark applications. This may be overridable by the user through command line parameters. The default value of the default GC (in case it is not provided in spark-defaults.conf) could be ParallelGC.
> I do not see this change required but I think it would benefit to the user experience.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org