You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Venkat Dabri <ve...@gmail.com> on 2018/10/16 15:21:56 UTC

Cached data not showing up in Storage tab

When I cache a variable the data never shows up in the
storage tab. The storage tab is always blank. I have tried it in
Zeppelin as well as spark-shell.

scala> val classCount = spark.read.parquet("s3:// ..../classCount")
scala> classCount.persist
scala> classCount.count

Nothing shows up in the Storage tab of either Zeppelin or spark-shell.
However, I have several running applications in production that does
show the data in cache. I am using Scala and Spark 2.2.1 in EMR. Any
workarounds to see the data in cache.

The problem is mentioned here:
https://forums.databricks.com/questions/117/why-is-my-rdd-not-showing-up-in-the-storage-tab-of.html
https://stackoverflow.com/questions/44792213/blank-storage-tab-in-spark-history-server

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org