You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jeff Zhang (JIRA)" <ji...@apache.org> on 2015/08/19 11:52:46 UTC
[jira] [Comment Edited] (SPARK-9195) RDD/Storage metrics don't
update cached partition counts when executors are removed/lost
[ https://issues.apache.org/jira/browse/SPARK-9195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702777#comment-14702777 ]
Jeff Zhang edited comment on SPARK-9195 at 8/19/15 9:52 AM:
------------------------------------------------------------
Find more issues on RDD/Storage UI
* Column "Size in Memory" and "Executor" is not correct when there's multiple replicas
* For the Data Distribution section, only the used executor should be displayed (currently all the executors will be displayed no matter whether it is used by this rdd)
* Size on Off-Head should also be displayed
Start to work on it.
was (Author: zjffdu):
Find more issues on RDD/Storage UI
* Column "Size in Memory" and "Executor" is not correct when there's multiple replicas
* For the Data Distribution section, only the used executor should be displayed (currently all the executors will be displayed no matter whether it is used by this rdd)
Start to work on it.
> RDD/Storage metrics don't update cached partition counts when executors are removed/lost
> ----------------------------------------------------------------------------------------
>
> Key: SPARK-9195
> URL: https://issues.apache.org/jira/browse/SPARK-9195
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.4.1
> Reporter: Ryan Williams
> Priority: Minor
>
> I have an RDD that was cached, and then i lost most of my executors (in this case it was due to the executors having been dynamically-allocated and timing out, but this applies when the executors are lost in general).
> The RDD page correctly shows that most of the RDD is no longer cached:
> !http://f.cl.ly/items/1i2C1v1H2o0i243f453P/Screen%20Shot%202015-07-20%20at%2011.46.11%20AM.png!
> (However, the "Cached Partitions" count at the top is incorrect).
> Additionally, the storage page still shows it as 100% cached:
> !http://f.cl.ly/items/3P3l440u0R3L3R0U1a2E/Screen%20Shot%202015-07-20%20at%2011.45.57%20AM.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org