You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sven Krasser (JIRA)" <ji...@apache.org> on 2015/01/29 00:23:34 UTC
[jira] [Commented] (SPARK-4049) Storage web UI "fraction cached"
shows as > 100%
[ https://issues.apache.org/jira/browse/SPARK-4049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296037#comment-14296037 ]
Sven Krasser commented on SPARK-4049:
-------------------------------------
I'm also seeing this for a 2x replicated RDD (StorageLevel.MEMORY_AND_DISK_2). I assume that means that most partitions are replicated twice and some three times?
That aside, I do not think it's a good idea to count overreplication towards that fraction in this way. As a user, when I see 100% on the UI, then I assume the RDD is fully cached. However, this could also mean that some partitions are missing (and need to be recomputed) and some are overreplicated.
> Storage web UI "fraction cached" shows as > 100%
> ------------------------------------------------
>
> Key: SPARK-4049
> URL: https://issues.apache.org/jira/browse/SPARK-4049
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.2.0
> Reporter: Josh Rosen
> Priority: Minor
>
> In the Storage tab of the Spark Web UI, I saw a case where the "Fraction Cached" was greater than 100%:
> !http://i.imgur.com/Gm2hEeL.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org