[jira] [Commented] (SPARK-1305) Support persisting RDD's directly to Tachyon

["Denis Serduik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14066465#comment-14066465 ] 

Denis Serduik commented on SPARK-1305:
--------------------------------------

I'm interesting in using this feature especially with SchemeRDD to be able cache intermediate  results.
Where I can find some code examples how to use it ? 
>From SparkContext's code I see the following: 

  private[spark] def persistRDD(rdd: RDD[_]) {
    persistentRdds(rdd.id) = rdd
  }

  /**
   * Unpersist an RDD from memory and/or disk storage
   */
  private[spark] def unpersistRDD(rddId: Int, blocking: Boolean = true) {
    env.blockManager.master.removeRdd(rddId, blocking)
    persistentRdds.remove(rddId)
    listenerBus.post(SparkListenerUnpersistRDD(rddId))
  }

So  persist don't put RDD into tachyonStore of blockmanager ? How does this feature work ? 


> Support persisting RDD's directly to Tachyon
> --------------------------------------------
>
>                 Key: SPARK-1305
>                 URL: https://issues.apache.org/jira/browse/SPARK-1305
>             Project: Spark
>          Issue Type: New Feature
>          Components: Block Manager
>            Reporter: Patrick Wendell
>            Assignee: Haoyuan Li
>            Priority: Blocker
>             Fix For: 1.0.0
>
>
> This is already an ongoing pull request - in a nutshell we want to support Tachyon as a storage level in Spark.



--
This message was sent by Atlassian JIRA
(v6.2#6252)