[jira] [Created] (SPARK-27704) Change default class loader to ParallelGC

["Mihaly Toth (JIRA)" <ji...@apache.org>]

Mihaly Toth created SPARK-27704:
-----------------------------------

             Summary: Change default class loader to ParallelGC
                 Key: SPARK-27704
                 URL: https://issues.apache.org/jira/browse/SPARK-27704
             Project: Spark
          Issue Type: Sub-task
          Components: Build
    Affects Versions: 3.0.0
            Reporter: Mihaly Toth


In JDK 11 the default class loader changed from ParallelGC to G1GC. Even though this gc performs better on pause times and interactivity, most of the tasks that need to be processed are more sensitive to throughput and the to the amount of memory. G1 sacrifices these to some extend to avoid the big pauses. As a result the user may perceive a regression compared to JDK 8. Even worse, the regression may not be limited to performance only but some jobs may start failing in case they do not fit into the memory they used to be happy with when running with previous JDK.

Some other kind of apps, like streaming ones, may rather use G1 because of their more interactive, more realtime needs.

With this jira it is proposed to have a configurable default GC for all spark applications. This may be overridable by the user through command line parameters. The default value of the default GC (in case it is not provided in spark-defaults.conf) could be ParallelGC.

I do not see this change required but I think it would benefit to the user experience.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org