You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by arosenberger <ad...@gmail.com> on 2014/02/20 14:57:03 UTC

Tachyon RDD Caching

Hi, in this pull request Spark adds the ability to store RDDs in off heap
memory via Tachyon: https://github.com/apache/incubator-spark/pull/468

I was wondering if the broadcast variable API would be opened up in a
similar manner? From the source of e.g. HttpBroadcast, it is currently hard
coded which would prevent Tachyon storage:
SparkEnv.get.blockManager.putSingle(blockId, value_,
StorageLevel.MEMORY_AND_DISK, false)

Thanks,
Adam



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Tachyon-RDD-Caching-tp1820.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.