You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Miklos Szurap (Jira)" <ji...@apache.org> on 2022/05/18 09:39:00 UTC

[jira] [Created] (SPARK-39222) Add JvmPauseMonitor to Spark History Server

Miklos Szurap created SPARK-39222:
-------------------------------------

             Summary: Add JvmPauseMonitor to Spark History Server
                 Key: SPARK-39222
                 URL: https://issues.apache.org/jira/browse/SPARK-39222
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 3.2.1, 3.0.0, 2.4.8, 2.0.0
            Reporter: Miklos Szurap


It is hard to understand what happens with Spark History Server when long JVM pauses (GC or host level pauses) are causing unresponsiveness. 
Similar to Hadoop's implementation (initially in HADOOP-9618) it would be beneficial to add the [JVMPauseMonitor|https://github.com/apache/hadoop/blob/f4b24c68e76df40d55258fc5391baabfa9ac362d/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/JvmPauseMonitor.java] to the HistoryServer (HistoryServerSuite.scala).
This will make GC pauses obvious in logs and lets administrators easily notice it and react in time - adjust configurations with increasing the SHS heap size.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org