You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2017/03/30 09:29:41 UTC
[jira] [Resolved] (SPARK-20150) Add permsize statistics for worker
memory which may be very useful for the memory usage assessment
[ https://issues.apache.org/jira/browse/SPARK-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-20150.
-------------------------------
Resolution: Not A Problem
From Spark 2.2, Java 8 is required, and there is no permgen anymore. I think we wouldn't add this, therefore.
> Add permsize statistics for worker memory which may be very useful for the memory usage assessment
> --------------------------------------------------------------------------------------------------
>
> Key: SPARK-20150
> URL: https://issues.apache.org/jira/browse/SPARK-20150
> Project: Spark
> Issue Type: Wish
> Components: Web UI
> Affects Versions: 2.0.2
> Reporter: Jinhua Fu
>
> It seems worker memory only be assigned to executor heap which is usually not enough for estimating the whole clauster memory usage,especially when memory becomes a bottleneck of the clauster.In many case,we found a executor's real memory usage was much larger than its heap size which make me have to check for every application's real memory expenditure.
> This can be improved by adding a mechanism for Non-Heap(permsize) statistics,only shown for extra memory usage which has no effect on the current worker memory allocation and statistics.The permsize can be obtained easily from executor java options.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org