[jira] [Commented] (SPARK-19289) UnCache Dataset using Name

["Sean Owen (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-19289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829982#comment-15829982 ] 

Sean Owen commented on SPARK-19289:
-----------------------------------

Names are not necessarily unique, and enforcing this would be a little bit of a behavior change.
I don't think there's any other context where RDDs and Datasets are retrieved by name. These are programmatic constructs. You have the handle to them in your driver program. It's not like SQL in this sense. I think it's fair easier to, well, use the actual objects!

> UnCache Dataset using Name
> --------------------------
>
>                 Key: SPARK-19289
>                 URL: https://issues.apache.org/jira/browse/SPARK-19289
>             Project: Spark
>          Issue Type: Wish
>          Components: Spark Core, SQL
>    Affects Versions: 2.1.0
>            Reporter: Kaushal Prajapati
>            Priority: Minor
>              Labels: features
>
> We can Cache and Uncache any table using its name in Spark Sql.
> {code}
> df.createTempView("myTable")
> sqlContext.cacheTable("myTable")
> sqlContext.uncacheTable("myTable")
> {code}
> Likewise if it is possible to have some kind of uniqueness for names in DataSets and an abstraction like the same that we have for tables. It would be very useful
> {code}
> scala> val df = sc.range(1,1000).toDF
> df: org.apache.spark.sql.DataFrame = [value: bigint]
> scala> df.setName("MyDataset")
> res0: df.type = MyDataset
> scala> df.cache
> res1: df.type = MyDataset
> sqlContext.getDataSet("MyDataset")
> sqlContext.uncacheDataSet("MyDataset")
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org