You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Zhang, Liye (JIRA)" <ji...@apache.org> on 2015/07/16 16:27:05 UTC

[jira] [Created] (SPARK-9103) Tracking spark's memory usage

Zhang, Liye created SPARK-9103:
----------------------------------

             Summary: Tracking spark's memory usage
                 Key: SPARK-9103
                 URL: https://issues.apache.org/jira/browse/SPARK-9103
             Project: Spark
          Issue Type: Umbrella
          Components: Spark Core, Web UI
            Reporter: Zhang, Liye


Currently spark only provides little memory usage information (RDD cache on webUI) for the executors. User have no idea on what is the memory consumption when they are running spark applications with a lot of memory used in spark executors. Especially when they encounter the OOM, it’s really hard to know what is the cause of the problem. So it would be helpful to give out the detail memory consumption information for each part of spark, so that user can clearly have a picture of where the memory is exactly used. 

The memory usage info to expose should include but not limited to shuffle, cache, network, serializer, etc.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org