[jira] [Created] (SPARK-25814) spark driver runs out of memory on org.apache.spark.util.kvstore.InMemoryStore

["Ruslan Dautkhanov (JIRA)" <ji...@apache.org>]

Ruslan Dautkhanov created SPARK-25814:
-----------------------------------------

             Summary: spark driver runs out of memory on org.apache.spark.util.kvstore.InMemoryStore
                 Key: SPARK-25814
                 URL: https://issues.apache.org/jira/browse/SPARK-25814
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.3.2, 2.2.2
            Reporter: Ruslan Dautkhanov
         Attachments: image-2018-10-23-14-06-53-722.png

 We're looking into issue when even huge spark driver memory gets eventually exhausted and GC makes driver stop responding.

Used jxray.com tool and found that most of driver heap is used by 

 
{noformat}
org.apache.spark.status.AppStatusStore
  -> org.apache.spark.status.ElementTrackingStore
    -> org.apache.spark.util.kvstore.InMemoryStore
 
{noformat}
 

 

Is there is a way to tune this particular spark driver's memory region down?

 

!image-2018-10-23-14-03-12-258.png!

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org