You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by clockfly <gi...@git.apache.org> on 2016/05/06 02:26:36 UTC
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
GitHub user clockfly opened a pull request:
https://github.com/apache/spark/pull/12945
[SPARK-15171][SQL]Deprecate registerTempTable and add dataset.createTempView
## What changes were proposed in this pull request?
Deprecates registerTempTable and add dataset.createTempView
## How was this patch tested?
Unit tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/clockfly/spark spark-15171
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12945.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12945
----
commit e4675b026f99687e0363aa5330335d03454360dc
Author: Sean Zhong <cl...@gmail.com>
Date: 2016-05-05T23:54:54Z
[SPARK-15171][SQL]Deprecate registerTempTable and add dataset.createTempView
Deprecates registerTempTable and add dataset.createTempView
Unit tests.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218074780
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58217/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217333434
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by yhuai <gi...@git.apache.org>.
Github user yhuai commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217942790
If possible, can you have a pr to just add the new API and deprecate the old API?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62969204
--- Diff: python/pyspark/sql/context.py ---
@@ -302,7 +302,7 @@ def registerDataFrameAsTable(self, df, tableName):
>>> sqlContext.registerDataFrameAsTable(df, "table1")
"""
- self.sparkSession.catalog.registerTable(df, tableName)
+ self.sparkSession._createTempView(tableName, df, replaceIfExists=True)
--- End diff --
can we use `df.createOrReplaceTempView("tableName")` here?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218083845
**[Test build #58213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58213/consoleFull)** for PR 12945 at commit [`3f834a4`](https://github.com/apache/spark/commit/3f834a427e89cd1f09525d528267245059f7173f).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218084789
**[Test build #58214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58214/consoleFull)** for PR 12945 at commit [`d53ca63`](https://github.com/apache/spark/commit/d53ca63e96718974fe771f67defcd1f0a230c294).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218684188
OK, I'm merging this to master and branch-2.0 since the last commit only changes deprecation message.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217333816
**[Test build #57949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57949/consoleFull)** for PR 12945 at commit [`8612976`](https://github.com/apache/spark/commit/8612976f43bc5757c733360b6381fc380004fff5).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217443221
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57986/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218488732
**[Test build #58377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58377/consoleFull)** for PR 12945 at commit [`3548d16`](https://github.com/apache/spark/commit/3548d16072f5a73fcc0a5a22829cfc8beedfd5c1).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218077501
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58219/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217353911
**[Test build #57964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57964/consoleFull)** for PR 12945 at commit [`e17bd6d`](https://github.com/apache/spark/commit/e17bd6d4a70536972e42fc15a81799730f49d31c).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217333976
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62800488
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala ---
@@ -175,13 +175,37 @@ abstract class Catalog {
options: Map[String, String]): DataFrame
/**
- * Drops the temporary table with the given table name in the catalog.
- * If the table has been cached before, then it will also be uncached.
+ * Drops the temporary view with the given view name in the catalog.
+ * If the view has been cached before, then it will also be uncached.
*
- * @param tableName the name of the table to be dropped.
+ * @param viewName the name of the view to be dropped.
* @since 2.0.0
*/
- def dropTempTable(tableName: String): Unit
+ def dropTempView(viewName: String): Unit
+
+
+ /**
+ * Creates a temporary view in the catalog with the given [[DataFrame]].
+ *
+ * @param viewName the name of the view to be created
+ * @df the DataFrame used to create the view.
+ *
+ * @throws AnalysisException if the view name already exists
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, df: DataFrame): Unit
--- End diff --
can we remove changes to Catalog.scala
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217348013
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217334522
**[Test build #57950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57950/consoleFull)** for PR 12945 at commit [`ac4e502`](https://github.com/apache/spark/commit/ac4e502dd1b6ea8f58f97e87935788cb9f0a7ee5).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218168191
**[Test build #58237 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58237/consoleFull)** for PR 12945 at commit [`08b744b`](https://github.com/apache/spark/commit/08b744bf731adcc6c6a8bd966972cb2d9f1a9251).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62607402
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2306,8 +2306,37 @@ class Dataset[T] private[sql](
* @group basic
* @since 1.6.0
*/
+ @deprecated("Use createTempView(tableName, replaceIfExists = true) instead.", "2.0.0")
def registerTempTable(tableName: String): Unit = {
- sparkSession.registerTable(toDF(), tableName)
+ createTempView(tableName, replaceIfExists = true)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
--- End diff --
SQLContext -> SparkSession
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218641930
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58432/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218642403
**[Test build #58433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58433/consoleFull)** for PR 12945 at commit [`16edd9d`](https://github.com/apache/spark/commit/16edd9dc5e914f60d74a6ffb015855a12f765a7a).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218101893
**[Test build #58223 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58223/consoleFull)** for PR 12945 at commit [`2c7b5f5`](https://github.com/apache/spark/commit/2c7b5f548bbaf7e2442df998725c1fe2727e83fb).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218102056
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58223/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218677966
**[Test build #58457 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58457/consoleFull)** for PR 12945 at commit [`dd52802`](https://github.com/apache/spark/commit/dd52802be56c10942d6b8ad9a6b18775b117d221).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by clockfly <gi...@git.apache.org>.
Github user clockfly commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217424914
There are two places I am not very sure:
1. Should we also deprecate the python and R api registerTempTable?
2. Which we allow replacing table or not for API createTempView(table)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62973873
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2303,13 +2303,39 @@ class Dataset[T] private[sql](
/**
* Registers this [[Dataset]] as a temporary table using the given name. The lifetime of this
- * temporary table is tied to the [[SQLContext]] that was used to create this Dataset.
+ * temporary table is tied to the [[SparkSession]] that was used to create this Dataset.
*
* @group basic
* @since 1.6.0
*/
+ @deprecated("Use createOrReplaceTempView(tableName) instead.", "2.0.0")
--- End diff --
Nit: `tableName` => `viewName`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218102053
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62799642
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala ---
@@ -175,13 +175,37 @@ abstract class Catalog {
options: Map[String, String]): DataFrame
/**
- * Drops the temporary table with the given table name in the catalog.
- * If the table has been cached before, then it will also be uncached.
+ * Drops the temporary view with the given view name in the catalog.
+ * If the view has been cached before, then it will also be uncached.
*
- * @param tableName the name of the table to be dropped.
+ * @param viewName the name of the view to be dropped.
* @since 2.0.0
*/
- def dropTempTable(tableName: String): Unit
+ def dropTempView(viewName: String): Unit
+
+
+ /**
+ * Creates a temporary view in the catalog with the given [[DataFrame]].
+ *
+ * @param viewName the name of the view to be created
+ * @df the DataFrame used to create the view.
+ *
+ * @throws AnalysisException if the view name already exists
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, df: DataFrame): Unit
+
+ /**
+ * Creates or replace a temporary view in the catalog with the given [[DataFrame]].
+ * Temporary view exist only during the lifetime of this instance of
+ * [[org.apache.spark.sql.SparkSession]].
+ *
+ * @param viewName the name of the view to be created/replaced.
+ * @df the DataFrame used to create the view.
+ * @since 2.0.0
+ */
+ def createOrReplaceTempView(viewName: String, df: DataFrame): Unit
--- End diff --
Why not adding a parameter `replace: Boolean` in `createTempView`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218702007
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217338675
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57953/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218382058
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218383401
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218069107
**[Test build #58213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58213/consoleFull)** for PR 12945 at commit [`3f834a4`](https://github.com/apache/spark/commit/3f834a427e89cd1f09525d528267245059f7173f).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217418602
**[Test build #57986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57986/consoleFull)** for PR 12945 at commit [`5bd77e9`](https://github.com/apache/spark/commit/5bd77e9838b343aa97ae43042804eea7f081e4d0).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218675439
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58449/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62973062
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala ---
@@ -674,6 +676,23 @@ class DatasetSuite extends QueryTest with SharedSQLContext {
}.getMessage
assert(message.contains("The 0th field of input row cannot be null"))
}
+
+ test("createTempView") {
+ val dataset = Seq(1, 2, 3).toDS()
+ dataset.createOrReplaceTempView("tempView")
+
+ noException should be thrownBy {
+ // Overrrides the existing temporary view with same name
+ dataset.createOrReplaceTempView("tempView")
+ }
--- End diff --
In general, we try to avoid using fancy ScalaTest assertion facilities like this one. In this case, a simple call to `createOrReplaceTempView` and a comment saying that no exception should be thrown is enough.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218206900
**[Test build #58237 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58237/consoleFull)** for PR 12945 at commit [`08b744b`](https://github.com/apache/spark/commit/08b744bf731adcc6c6a8bd966972cb2d9f1a9251).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62799928
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala ---
@@ -175,13 +175,37 @@ abstract class Catalog {
options: Map[String, String]): DataFrame
/**
- * Drops the temporary table with the given table name in the catalog.
- * If the table has been cached before, then it will also be uncached.
+ * Drops the temporary view with the given view name in the catalog.
+ * If the view has been cached before, then it will also be uncached.
*
- * @param tableName the name of the table to be dropped.
+ * @param viewName the name of the view to be dropped.
* @since 2.0.0
*/
- def dropTempTable(tableName: String): Unit
+ def dropTempView(viewName: String): Unit
+
+
+ /**
+ * Creates a temporary view in the catalog with the given [[DataFrame]].
+ *
+ * @param viewName the name of the view to be created
+ * @df the DataFrame used to create the view.
+ *
+ * @throws AnalysisException if the view name already exists
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, df: DataFrame): Unit
+
+ /**
+ * Creates or replace a temporary view in the catalog with the given [[DataFrame]].
+ * Temporary view exist only during the lifetime of this instance of
+ * [[org.apache.spark.sql.SparkSession]].
+ *
+ * @param viewName the name of the view to be created/replaced.
+ * @df the DataFrame used to create the view.
+ * @since 2.0.0
+ */
+ def createOrReplaceTempView(viewName: String, df: DataFrame): Unit
--- End diff --
see discussion above. It's a bad idea in general to have boolean arguments in APIs.
For example, if you look at `createTempView("abcd", false)`, it is unclear what the false means.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218642407
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58433/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217828967
I think we should also deprecate `Catalog.dropTempTable` and `SQLContext.dropTempTable` by `Catalog.dropTempView` and `SQLContext.dropTempView`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217333310
**[Test build #57947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57947/consoleFull)** for PR 12945 at commit [`e4675b0`](https://github.com/apache/spark/commit/e4675b026f99687e0363aa5330335d03454360dc).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218081294
**[Test build #58223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58223/consoleFull)** for PR 12945 at commit [`2c7b5f5`](https://github.com/apache/spark/commit/2c7b5f548bbaf7e2442df998725c1fe2727e83fb).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217348014
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57956/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62898236
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala ---
@@ -175,13 +175,37 @@ abstract class Catalog {
options: Map[String, String]): DataFrame
/**
- * Drops the temporary table with the given table name in the catalog.
- * If the table has been cached before, then it will also be uncached.
+ * Drops the temporary view with the given view name in the catalog.
+ * If the view has been cached before, then it will also be uncached.
*
- * @param tableName the name of the table to be dropped.
+ * @param viewName the name of the view to be dropped.
* @since 2.0.0
*/
- def dropTempTable(tableName: String): Unit
+ def dropTempView(viewName: String): Unit
+
+
+ /**
+ * Creates a temporary view in the catalog with the given [[DataFrame]].
+ *
+ * @param viewName the name of the view to be created
+ * @df the DataFrame used to create the view.
+ *
+ * @throws AnalysisException if the view name already exists
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, df: DataFrame): Unit
--- End diff --
As it stands right now, Catalog.scala doesn't have any other interface for creating tables, so I'd prefer not having them for views for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218144359
**[Test build #58234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58234/consoleFull)** for PR 12945 at commit [`f065496`](https://github.com/apache/spark/commit/f0654969f344a0e013cda180452525acf14c73c9).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218383367
**[Test build #58337 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58337/consoleFull)** for PR 12945 at commit [`e01b3e8`](https://github.com/apache/spark/commit/e01b3e811cd8229822aebb18621e926d7649448b).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217443219
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218084957
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218406840
**[Test build #58347 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58347/consoleFull)** for PR 12945 at commit [`98e5c90`](https://github.com/apache/spark/commit/98e5c90563cfce50b1f1831d739d125ce73aec6a).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218697629
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58456/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217335041
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57950/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218070503
**[Test build #58214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58214/consoleFull)** for PR 12945 at commit [`d53ca63`](https://github.com/apache/spark/commit/d53ca63e96718974fe771f67defcd1f0a230c294).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218144669
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218642223
**[Test build #58433 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58433/consoleFull)** for PR 12945 at commit [`16edd9d`](https://github.com/apache/spark/commit/16edd9dc5e914f60d74a6ffb015855a12f765a7a).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218383404
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58337/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218103213
**[Test build #58226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58226/consoleFull)** for PR 12945 at commit [`cc74aaa`](https://github.com/apache/spark/commit/cc74aaa56577f12453dc5183c8713234950ba7f3).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217333435
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57947/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218575937
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58390/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218084008
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218697621
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218641927
**[Test build #58432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58432/consoleFull)** for PR 12945 at commit [`1d1eeb0`](https://github.com/apache/spark/commit/1d1eeb091ffd16130047d08957b9d199a21564ef).
* This patch **fails Python style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62607434
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2306,8 +2306,37 @@ class Dataset[T] private[sql](
* @group basic
* @since 1.6.0
*/
+ @deprecated("Use createTempView(tableName, replaceIfExists = true) instead.", "2.0.0")
def registerTempTable(tableName: String): Unit = {
- sparkSession.registerTable(toDF(), tableName)
+ createTempView(tableName, replaceIfExists = true)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if the view name already exists
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String): Unit = {
+ createTempView(viewName, replaceIfExists = false)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if replaceIfExists flag is false and the view name already exists.
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, replaceIfExists: Boolean): Unit = {
--- End diff --
how about we just rename this to createOrReplaceTempView, and remove the boolean flag?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218522426
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58380/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217372513
**[Test build #57964 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57964/consoleFull)** for PR 12945 at commit [`e17bd6d`](https://github.com/apache/spark/commit/e17bd6d4a70536972e42fc15a81799730f49d31c).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218077482
**[Test build #58219 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58219/consoleFull)** for PR 12945 at commit [`d372575`](https://github.com/apache/spark/commit/d3725753bef6d94f542a096250899084c2c3987a).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218074772
**[Test build #58217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58217/consoleFull)** for PR 12945 at commit [`59654e6`](https://github.com/apache/spark/commit/59654e6271b42573d2de58355cdf2d0371649294).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217338674
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218126141
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218682289
LGTM except for one minor comment for the deprecation warning message.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62972608
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala ---
@@ -175,13 +175,37 @@ abstract class Catalog {
options: Map[String, String]): DataFrame
/**
- * Drops the temporary table with the given table name in the catalog.
- * If the table has been cached before, then it will also be uncached.
+ * Drops the temporary view with the given view name in the catalog.
+ * If the view has been cached before, then it will also be uncached.
*
- * @param tableName the name of the table to be dropped.
+ * @param viewName the name of the view to be dropped.
* @since 2.0.0
*/
- def dropTempTable(tableName: String): Unit
+ def dropTempView(viewName: String): Unit
+
+
+ /**
+ * Creates a temporary view in the catalog with the given [[DataFrame]].
+ *
+ * @param viewName the name of the view to be created
+ * @df the DataFrame used to create the view.
+ *
+ * @throws AnalysisException if the view name already exists
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, df: DataFrame): Unit
--- End diff --
I see, it's kinda like the `SparkSession.read` and `Dataset.write` pair.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218382060
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58335/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218508867
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58377/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218673040
The PR description is inconsistent with changes made in the PR now. Please update.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/12945
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218522357
**[Test build #58380 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58380/consoleFull)** for PR 12945 at commit [`302ca72`](https://github.com/apache/spark/commit/302ca7211496872970ee8b462bde112f2a51b8da).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217442978
**[Test build #57986 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57986/consoleFull)** for PR 12945 at commit [`5bd77e9`](https://github.com/apache/spark/commit/5bd77e9838b343aa97ae43042804eea7f081e4d0).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217347997
**[Test build #57956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57956/consoleFull)** for PR 12945 at commit [`029ce73`](https://github.com/apache/spark/commit/029ce73aab1d57f85a506371a6989a5e3405a85c).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217335037
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by clockfly <gi...@git.apache.org>.
Github user clockfly commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62802730
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala ---
@@ -175,13 +175,37 @@ abstract class Catalog {
options: Map[String, String]): DataFrame
/**
- * Drops the temporary table with the given table name in the catalog.
- * If the table has been cached before, then it will also be uncached.
+ * Drops the temporary view with the given view name in the catalog.
+ * If the view has been cached before, then it will also be uncached.
*
- * @param tableName the name of the table to be dropped.
+ * @param viewName the name of the view to be dropped.
* @since 2.0.0
*/
- def dropTempTable(tableName: String): Unit
+ def dropTempView(viewName: String): Unit
+
+
+ /**
+ * Creates a temporary view in the catalog with the given [[DataFrame]].
+ *
+ * @param viewName the name of the view to be created
+ * @df the DataFrame used to create the view.
+ *
+ * @throws AnalysisException if the view name already exists
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, df: DataFrame): Unit
--- End diff --
@rxin, we don't want to change Catalog.scala?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217341678
**[Test build #57956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57956/consoleFull)** for PR 12945 at commit [`029ce73`](https://github.com/apache/spark/commit/029ce73aab1d57f85a506371a6989a5e3405a85c).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218701718
**[Test build #58457 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58457/consoleFull)** for PR 12945 at commit [`dd52802`](https://github.com/apache/spark/commit/dd52802be56c10942d6b8ad9a6b18775b117d221).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218702011
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58457/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218077499
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218385649
**[Test build #58347 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58347/consoleFull)** for PR 12945 at commit [`98e5c90`](https://github.com/apache/spark/commit/98e5c90563cfce50b1f1831d739d125ce73aec6a).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218407061
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58347/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218075958
**[Test build #58219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58219/consoleFull)** for PR 12945 at commit [`d372575`](https://github.com/apache/spark/commit/d3725753bef6d94f542a096250899084c2c3987a).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218126142
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58226/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218642406
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218074777
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217826714
For question 1, I think we should also deprecate corresponding API in PySpark and SparkR. For question 2, I don't get it...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218575660
**[Test build #58390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58390/consoleFull)** for PR 12945 at commit [`deecc1a`](https://github.com/apache/spark/commit/deecc1a2d737be4f58e83a05162fb9570e0f6695).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62969319
--- Diff: python/pyspark/sql/session.py ---
@@ -255,6 +255,19 @@ def range(self, start, end=None, step=1, numPartitions=None):
return DataFrame(jdf, self._wrapped)
+ def _createTempView(self, viewName, dataFrame, replaceIfExists):
--- End diff --
Why do we add this method?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217333974
**[Test build #57949 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57949/consoleFull)** for PR 12945 at commit [`8612976`](https://github.com/apache/spark/commit/8612976f43bc5757c733360b6381fc380004fff5).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by clockfly <gi...@git.apache.org>.
Github user clockfly commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62617764
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2306,8 +2306,37 @@ class Dataset[T] private[sql](
* @group basic
* @since 1.6.0
*/
+ @deprecated("Use createTempView(tableName, replaceIfExists = true) instead.", "2.0.0")
def registerTempTable(tableName: String): Unit = {
- sparkSession.registerTable(toDF(), tableName)
+ createTempView(tableName, replaceIfExists = true)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if the view name already exists
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String): Unit = {
+ createTempView(viewName, replaceIfExists = false)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if replaceIfExists flag is false and the view name already exists.
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, replaceIfExists: Boolean): Unit = {
--- End diff --
@rxin, there is a little difference between these two.
createTempView(viewName, replace=false) will throw exception when view name already exists.
createOrReplaceTempView will **NOT** throw exception when view name exists.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218641929
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218407057
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218701473
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218084959
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58214/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218660928
**[Test build #58449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58449/consoleFull)** for PR 12945 at commit [`a9bdd5b`](https://github.com/apache/spark/commit/a9bdd5b680f22e17cb9cda8c234ca57ae92e80f2).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218683343
**[Test build #58459 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58459/consoleFull)** for PR 12945 at commit [`b024e99`](https://github.com/apache/spark/commit/b024e998184ad78202d22c661ec0ea20125b4cab).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218382014
**[Test build #58335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58335/consoleFull)** for PR 12945 at commit [`72d7103`](https://github.com/apache/spark/commit/72d71034f4e6723b81f77b870839c1613d340fe7).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218522423
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218207250
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218543201
**[Test build #58390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58390/consoleFull)** for PR 12945 at commit [`deecc1a`](https://github.com/apache/spark/commit/deecc1a2d737be4f58e83a05162fb9570e0f6695).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218373779
**[Test build #58335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58335/consoleFull)** for PR 12945 at commit [`72d7103`](https://github.com/apache/spark/commit/72d71034f4e6723b81f77b870839c1613d340fe7).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62799493
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala ---
@@ -175,13 +175,37 @@ abstract class Catalog {
options: Map[String, String]): DataFrame
/**
- * Drops the temporary table with the given table name in the catalog.
- * If the table has been cached before, then it will also be uncached.
+ * Drops the temporary view with the given view name in the catalog.
+ * If the view has been cached before, then it will also be uncached.
*
- * @param tableName the name of the table to be dropped.
+ * @param viewName the name of the view to be dropped.
* @since 2.0.0
*/
- def dropTempTable(tableName: String): Unit
+ def dropTempView(viewName: String): Unit
+
+
+ /**
+ * Creates a temporary view in the catalog with the given [[DataFrame]].
+ *
+ * @param viewName the name of the view to be created
+ * @df the DataFrame used to create the view.
--- End diff --
nit: `@param df`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218508796
**[Test build #58377 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58377/consoleFull)** for PR 12945 at commit [`3548d16`](https://github.com/apache/spark/commit/3548d16072f5a73fcc0a5a22829cfc8beedfd5c1).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218207251
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58237/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218144672
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58234/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218084010
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58213/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218701476
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58459/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218676326
**[Test build #58456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58456/consoleFull)** for PR 12945 at commit [`977e01d`](https://github.com/apache/spark/commit/977e01dfa4ee950c492819ca2bddf1124944d297).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218641669
**[Test build #58432 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58432/consoleFull)** for PR 12945 at commit [`1d1eeb0`](https://github.com/apache/spark/commit/1d1eeb091ffd16130047d08957b9d199a21564ef).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218508864
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218374585
**[Test build #58337 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58337/consoleFull)** for PR 12945 at commit [`e01b3e8`](https://github.com/apache/spark/commit/e01b3e811cd8229822aebb18621e926d7649448b).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62618235
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2306,8 +2306,37 @@ class Dataset[T] private[sql](
* @group basic
* @since 1.6.0
*/
+ @deprecated("Use createTempView(tableName, replaceIfExists = true) instead.", "2.0.0")
def registerTempTable(tableName: String): Unit = {
- sparkSession.registerTable(toDF(), tableName)
+ createTempView(tableName, replaceIfExists = true)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if the view name already exists
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String): Unit = {
+ createTempView(viewName, replaceIfExists = false)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if replaceIfExists flag is false and the view name already exists.
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, replaceIfExists: Boolean): Unit = {
--- End diff --
But we already have createTempView.
What I'm saying is to define two functions:
1. createTempView(viewName: String)
2. createOrReplaceTempView(viewName: String)
Then it is very obvious when looking at their usage to know what the behavior is. Otherwise we will run into the issue of not knowing what the 2nd parameter mean unless somebody looks up the documentation, e.g. `createTempView("abcd", false)` - what does the false mean?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by clockfly <gi...@git.apache.org>.
Github user clockfly commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62618388
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2306,8 +2306,37 @@ class Dataset[T] private[sql](
* @group basic
* @since 1.6.0
*/
+ @deprecated("Use createTempView(tableName, replaceIfExists = true) instead.", "2.0.0")
def registerTempTable(tableName: String): Unit = {
- sparkSession.registerTable(toDF(), tableName)
+ createTempView(tableName, replaceIfExists = true)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if the view name already exists
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String): Unit = {
+ createTempView(viewName, replaceIfExists = false)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if replaceIfExists flag is false and the view name already exists.
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, replaceIfExists: Boolean): Unit = {
--- End diff --
OK, will update the code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217372543
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by clockfly <gi...@git.apache.org>.
Github user clockfly commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218374422
Rebased on latest trunk.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217372544
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57964/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217338558
**[Test build #57953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57953/consoleFull)** for PR 12945 at commit [`0bcbe8c`](https://github.com/apache/spark/commit/0bcbe8ce21ad3b1d295fda3e95aa1fbdfb60320c).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217333977
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57949/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218675297
**[Test build #58449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58449/consoleFull)** for PR 12945 at commit [`a9bdd5b`](https://github.com/apache/spark/commit/a9bdd5b680f22e17cb9cda8c234ca57ae92e80f2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217333432
**[Test build #57947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57947/consoleFull)** for PR 12945 at commit [`e4675b0`](https://github.com/apache/spark/commit/e4675b026f99687e0363aa5330335d03454360dc).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218701322
**[Test build #58459 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58459/consoleFull)** for PR 12945 at commit [`b024e99`](https://github.com/apache/spark/commit/b024e998184ad78202d22c661ec0ea20125b4cab).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218144666
**[Test build #58234 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58234/consoleFull)** for PR 12945 at commit [`f065496`](https://github.com/apache/spark/commit/f0654969f344a0e013cda180452525acf14c73c9).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218575935
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218074357
**[Test build #58217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58217/consoleFull)** for PR 12945 at commit [`59654e6`](https://github.com/apache/spark/commit/59654e6271b42573d2de58355cdf2d0371649294).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218504660
**[Test build #58380 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58380/consoleFull)** for PR 12945 at commit [`302ca72`](https://github.com/apache/spark/commit/302ca7211496872970ee8b462bde112f2a51b8da).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218675438
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62866763
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/catalog/Catalog.scala ---
@@ -175,13 +175,37 @@ abstract class Catalog {
options: Map[String, String]): DataFrame
/**
- * Drops the temporary table with the given table name in the catalog.
- * If the table has been cached before, then it will also be uncached.
+ * Drops the temporary view with the given view name in the catalog.
+ * If the view has been cached before, then it will also be uncached.
*
- * @param tableName the name of the table to be dropped.
+ * @param viewName the name of the view to be dropped.
* @since 2.0.0
*/
- def dropTempTable(tableName: String): Unit
+ def dropTempView(viewName: String): Unit
+
+
+ /**
+ * Creates a temporary view in the catalog with the given [[DataFrame]].
+ *
+ * @param viewName the name of the view to be created
+ * @df the DataFrame used to create the view.
+ *
+ * @throws AnalysisException if the view name already exists
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String, df: DataFrame): Unit
--- End diff --
@rxin Do you suggest we'd better make this PR more surgical and handle other changes in follow-up PRs?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218125970
**[Test build #58226 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58226/consoleFull)** for PR 12945 at commit [`cc74aaa`](https://github.com/apache/spark/commit/cc74aaa56577f12453dc5183c8713234950ba7f3).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218697421
**[Test build #58456 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58456/consoleFull)** for PR 12945 at commit [`977e01d`](https://github.com/apache/spark/commit/977e01dfa4ee950c492819ca2bddf1124944d297).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217338672
**[Test build #57953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57953/consoleFull)** for PR 12945 at commit [`0bcbe8c`](https://github.com/apache/spark/commit/0bcbe8ce21ad3b1d295fda3e95aa1fbdfb60320c).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-217335030
**[Test build #57950 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57950/consoleFull)** for PR 12945 at commit [`ac4e502`](https://github.com/apache/spark/commit/ac4e502dd1b6ea8f58f97e87935788cb9f0a7ee5).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL][WIP]Deprecate registerTempT...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/12945#discussion_r62607407
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2306,8 +2306,37 @@ class Dataset[T] private[sql](
* @group basic
* @since 1.6.0
*/
+ @deprecated("Use createTempView(tableName, replaceIfExists = true) instead.", "2.0.0")
def registerTempTable(tableName: String): Unit = {
- sparkSession.registerTable(toDF(), tableName)
+ createTempView(tableName, replaceIfExists = true)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
+ *
+ * @throws AnalysisException if the view name already exists
+ *
+ * @group basic
+ * @since 2.0.0
+ */
+ @throws[AnalysisException]
+ def createTempView(viewName: String): Unit = {
+ createTempView(viewName, replaceIfExists = false)
+ }
+
+ /**
+ * Creates a temporary view using the given name. The lifetime of this
+ * temporary view is tied to the [[SQLContext]] that was used to create this Dataset.
--- End diff --
SQLContext -> SparkSession
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-15171][SQL]Deprecate registerTempTable ...
Posted by clockfly <gi...@git.apache.org>.
Github user clockfly commented on the pull request:
https://github.com/apache/spark/pull/12945#issuecomment-218215578
There will be additional PRs to replace the reference of deprecated API.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org