[jira] [Created] (SPARK-30945) Preserve JVM information when executor heartbeat timeout

["wangzhun (Jira)" <ji...@apache.org>]

wangzhun created SPARK-30945:
--------------------------------

             Summary: Preserve JVM information when executor heartbeat timeout
                 Key: SPARK-30945
                 URL: https://issues.apache.org/jira/browse/SPARK-30945
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
    Affects Versions: 2.3.0
            Reporter: wangzhun


In practice, it was found that executor has a gc loop state, which will cause gc to spend a lot of time. Usually, this situation does not cause OOM, but causes the connection driver to timeout.

First, we need to check the status of the gc loop,by calculating the cumulative time of GC abnormal state.By setting a reasonable threshold, save the JVM Heap before the executor heartbeat timeout


Of course, we can also detect the unhealthy state of the driver



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org