You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/23 03:55:22 UTC
[GitHub] [spark] LantaoJin opened a new pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
LantaoJin opened a new pull request #28901:
URL: https://github.com/apache/spark/pull/28901
### What changes were proposed in this pull request?
Many databases and data warehouse SQL engines support temporary tables. A temporary table, as its named implied, is a short-lived table that its life will be only for current session.
[Hive Temporary Table](https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.0.1/using-hiveql/content/hive_create_a_hive_temporary_table.html)
[Teradata Volatile Table](https://docs.teradata.com/reader/rgAb27O_xRmMVc_aQq2VGw/mpJF1z_vSlpMbZYxFmRJfA)
[PostgreSQL Temporary Table](https://www.postgresql.org/docs/12/sql-createtable.html)
In Spark, there is no temporary table. the DDL “CREATE TEMPORARY TABLE AS SELECT” will create a temporary view. A temporary view is totally different with a temporary table.
This ticket to support Spark native temporary table. More details are described in [DESIGN DOCS](https://docs.google.com/document/d/1RS4Q3VbxlZ_Yy0fdWgTJ-k0QxFd1dToCqpLAYvIJ34U/edit?usp=sharing)
### Why are the changes needed?
A temporary view is just a VIEW. It doesn’t materialize data in storage. So it has below shortage:
1. View will not give improved performance. Materialize intermediate data in temporary tables for a complex query will accurate queries, especially in an ETL pipeline.
2. View which calls other views can cause severe performance issues. Even, executing a very complex view may fail in Spark.
3. Temporary view has no database namespace. In some complex ETL pipelines or data warehouse applications, without database prefix is not convenient. It needs some tables which only used in current session.
### Does this PR introduce _any_ user-facing change?
YES.
```sql
CREATE TEMPORARY TABLE tt1 AS SELECT ..
```
before the patch, it will create a local temporary VIEW. After this patch, it will create a temporary table.
```sql
CREATE TEMPORARY TABLE tt1 USING ..
```
before the patch, it will throw exception. After this patch, it will create a temporary table.
### How was this patch tested?
Add unit tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654616006
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648548339
**[Test build #124447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124447/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648544945
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648610389
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648609864
**[Test build #124463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124463/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647945906
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rxin commented on a change in pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
rxin commented on a change in pull request #28901:
URL: https://github.com/apache/spark/pull/28901#discussion_r449984452
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/StaticSQLConf.scala
##########
@@ -226,4 +227,22 @@ object StaticSQLConf {
.version("3.0.0")
.intConf
.createWithDefault(100)
+
+ val SPARK_SCRATCH_DIR =
+ buildStaticConf("spark.scratchdir")
+ .doc("Scratch space for Spark temporary table and so on. Similar with hive.exec.scratchdir")
Review comment:
let's not bring up hive here. Slowly nobody will care about Hive.
Also this should be spark.sql.scratchdir?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] rxin commented on a change in pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
rxin commented on a change in pull request #28901:
URL: https://github.com/apache/spark/pull/28901#discussion_r449984265
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
##########
@@ -3030,6 +3031,215 @@ abstract class DDLSuite extends QueryTest with SQLTestUtils {
}
}
}
+
+ test("create temporary table using data source") {
Review comment:
maybe create a new suite for these?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654617568
**[Test build #125178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125178/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647952591
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650708334
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654589955
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648635663
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648549575
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650747302
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650708338
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124594/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647946382
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650714043
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-657972128
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654533646
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648539644
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654673849
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648539040
**[Test build #124444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124444/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654763077
**[Test build #125195 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125195/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647946391
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124394/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654533286
**[Test build #125153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125153/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647964773
**[Test build #124400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124400/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647952061
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648543831
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650713934
**[Test build #124600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124600/testReport)** for PR 28901 at commit [`62e8ac9`](https://github.com/apache/spark/commit/62e8ac9d2b61c0aee684133418fec1bfae164930).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650700128
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654588832
**[Test build #125153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125153/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gatorsmile commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
gatorsmile commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654013787
For a proper support, this requires more discussions about the semantics.
Also, we need to list the expected behaviors for all the statements listed in https://spark.apache.org/docs/latest/sql-ref-syntax.html .
So far, this PR and design doc does not have the corresponding contents.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654589961
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125153/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-657973941
**[Test build #125805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125805/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654644370
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650700037
**[Test build #124594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124594/testReport)** for PR 28901 at commit [`6d3274f`](https://github.com/apache/spark/commit/6d3274f08c1c81262c8b0c21aa133a04e31c6796).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650700037
**[Test build #124594 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124594/testReport)** for PR 28901 at commit [`6d3274f`](https://github.com/apache/spark/commit/6d3274f08c1c81262c8b0c21aa133a04e31c6796).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654677094
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648635675
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124463/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648544578
**[Test build #124446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124446/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648539644
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647895663
**[Test build #124382 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124382/testReport)** for PR 28901 at commit [`f3f4e15`](https://github.com/apache/spark/commit/f3f4e15cfff650d55dec50b77e52b2a7d2142014).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `class TempViewAlreadyExistsException(table: String)`
* `class TempTableAlreadyExistsException(table: TableIdentifier)`
* `class TempTablePartitionUnsupportedException(table: TableIdentifier)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648539649
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124444/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654148439
If I write the output to a temp location and then create a temp view, is it similar to the temp table? Except that temp table can be removed when the session terminates.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647895692
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124382/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654644370
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650714043
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654212084
Let me clarify my previous comment a little bit: If I write the output to a temp location and then create a temp view **to read from this temp location**, ...
I agree with your use case, but I'm bit worried about adding a new big API (temp table) if there are easy workarounds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654198375
> If I write the output to a temp location and then create a temp view, is it similar to the temp table? Except that temp table can be removed when the session terminates.
There is no path and materialized data for temp view. So the answer is no. You can simply treate a `temporary table` as a Spark permanent data source table which will be dropped automatically when the session closed. So the implementation is not complex. But the user cases of temporary table are more than it looks. A permanent metastore table needs more maintenance, creating a permanent metastore table needs write permission of database it related and folder permission of storage. For many ad-hoc user case (OLAP), a user may only have limited permission like read. So current, users use `temporary view` to implement their complicated queries. View will not give improved performance without data materialzation. So if user can create `temporary table`, no need to grant write permission in production databases, it is very convenience for users.
Imaging the user case like below:
A databricks runtime user login to the notebook. He/She will write some statements to do an analysis job. Maybe in databricks runtime, user has r/w permissions on his/her own default space (we call it workspace), but no write permission on production database (for example, database "dw"). Without `temporary table`, they may use temporary view in their SQL statements. Or they can create a temporary workspace/database (for example, database named "tony_work"), and create permanent tables in "tony_work" then drop them all when they logout (if they can logout without failure). But users may want to share their scripts (above SQL statements) to another user or batch account. He/She has to change their scripts since the batch account or another user don't have the write permission on database "tony_ work". So in our production, the `temporary table` feature is wildly used, especially for the Teradata users who migrated to Spark. @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on a change in pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on a change in pull request #28901:
URL: https://github.com/apache/spark/pull/28901#discussion_r450546734
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
##########
@@ -3030,6 +3031,215 @@ abstract class DDLSuite extends QueryTest with SQLTestUtils {
}
}
}
+
+ test("create temporary table using data source") {
Review comment:
sure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650779166
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-658009628
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654047474
> For a proper support, this requires more discussions about the semantics.
>
> Also, we need to list the expected behaviors for all the statements listed in https://spark.apache.org/docs/latest/sql-ref-syntax.html .
>
> So far, this PR and design doc does not have the corresponding contents.
Sure, if you could tell me more about what do we need to discuss and what details should be written in documentation, that would be very helpful to me. About the concept of "temporary table", I think it is widly used in database domain: MySQL, PostgreSQL, Oracle etc, also in data warehouse domain: Hive, Teradata etc. Even though their implementations and grammar maybe different more or less, the purposes are similar in my opinion. This implementation and grammar of `Spark temporary table` mainly references [Hive](https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.0.1/using-hiveql/content/hive_create_a_hive_temporary_table.html) and [PostgreSQL](https://www.postgresql.org/docs/12/sql-createtable.html). They are basicly same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650728621
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650778655
**[Test build #124603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124603/testReport)** for PR 28901 at commit [`62e8ac9`](https://github.com/apache/spark/commit/62e8ac9d2b61c0aee684133418fec1bfae164930).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648539040
**[Test build #124444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124444/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648635663
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648549564
**[Test build #124447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124447/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647952061
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-658009630
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125805/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647963940
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648548339
**[Test build #124447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124447/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648537549
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654617568
**[Test build #125178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125178/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648544578
**[Test build #124446 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124446/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648544945
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin edited a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin edited a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654224633
> If I write the output to a temp location and then create a temp view to read from this temp location, ...
Ah, I knew your meaning now, using `CREATE TEMP VIEW USING` command. I think it's still tricky. First, platform needs to provide the "temp" location to storage users' data which are able to clean all untracked files. It's hard to do that. Second, "CREATE TEMP VIEW USING" command lack the view definition. In most cases, immutable `temporary table` is enough. After the temporary created with `AS SELECT ...`, less insert/overwrite operations will perform on it. `CREATE TEMP VIEW USING` command without `AS SELECT`, most practises seem to `insert/overwrite` on it. What's more, the storage folder permission is still a problem in a ACL enabled platform (even Spark code didn't import ACL, many companies build their platforms similar with Databricks Runtime' ACL)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654533286
**[Test build #125153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125153/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650747889
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648609864
**[Test build #124463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124463/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647893416
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648635486
**[Test build #124463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124463/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650728621
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648545390
**[Test build #124446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124446/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647964773
**[Test build #124400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124400/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-658009289
**[Test build #125805 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125805/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654589955
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647952575
**[Test build #124397 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124397/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654224633
> If I write the output to a temp location and then create a temp view to read from this temp location, ...
Ah, I knew your meaning now, using `CREATE TEMP VIEW USING` command. I think it's still tricky. First, platform needs to provide the "temp" location to storage users' data which are able to clean all untracked files. It's hard to do that. Second, "CREATE TEMP VIEW USING" command lack the view definition. In most cases, immutable `temporary table` is enough. After the temporary created with `AS SELECT ...`, less insert/overwrite operations will perform on it. `CREATE TEMP VIEW USING` command without `AS SELECT`, a good practise seems to `insert/overwrite` it then. What's more, the storage folder permission is still a problem in a ACL enabled platform (even Spark code didn't import ACL, many companies build their platforms similar with Databricks Runtime)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654533646
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-662859641
> how about to use `CACHE TABLE` command to do that?
I think you mean `CACHE VIEW` since `CACHE TABLE` still needs to `DROP TABLE` manually.
There are four reasons to build `temporary table` instead of `cache a temporary view`:
1. The intermediate table which users want to create as a temporary table are always very large. To avoid OOM, user has to use `CACHE TABLE viewname OPTIONS('storageLevel', 'disk_only')`. It's not friend to SQL users. Users confuse what 'storageLevel' and 'disk_only' are.
2. View is dynamic and table is static. Whatever the underly detail tables changes, accessing a view should always access the latest data. So when the underly detail tables of a view changed, Spark will recache all data to executors's local disks again. For a large intermediate table, this is not performance friendly.
3. The storages between cached view and temporary table are different. Cache command stores the block in executors local disk which managed by `blockManager`, and data of temporary table is stored in external storage like HDFS. Local disks in executors are very limited and not easy to scale out. Besides, the data in HDFS can be organized by Parquet file format, this can highly benefits `Scan` operation and predicates pushdown.
4. The accuracy of table statistics for cached view is easily expired. IIUC, table statistics for cached view are calculated when the cache operation occurs. When the data of under details tables changed, the statistics of a cached view won't be updated. So some optimization like AQE cannot work correctly. But to a temporary table, no problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654677094
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650708334
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650747889
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648539633
**[Test build #124444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124444/testReport)** for PR 28901 at commit [`5f23917`](https://github.com/apache/spark/commit/5f23917a1e7c9677c8cf87bf2fd3bd192ec7baea).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-657974354
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654764068
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125195/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647951434
**[Test build #124397 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124397/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650747727
**[Test build #124603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124603/testReport)** for PR 28901 at commit [`62e8ac9`](https://github.com/apache/spark/commit/62e8ac9d2b61c0aee684133418fec1bfae164930).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647945315
**[Test build #124394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124394/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650728526
**[Test build #124600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124600/testReport)** for PR 28901 at commit [`62e8ac9`](https://github.com/apache/spark/commit/62e8ac9d2b61c0aee684133418fec1bfae164930).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648050500
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648610389
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648050513
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124400/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647950482
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647951434
**[Test build #124397 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124397/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647893150
**[Test build #124382 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124382/testReport)** for PR 28901 at commit [`f3f4e15`](https://github.com/apache/spark/commit/f3f4e15cfff650d55dec50b77e52b2a7d2142014).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648547350
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654644380
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/125178/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647965443
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-653281307
@gatorsmile @cloud-fan Current implementation is not complex. Any comments?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-658009628
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648548682
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648549575
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-657974354
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647893416
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-657973941
**[Test build #125805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125805/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654764058
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654676289
**[Test build #125195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125195/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650713934
**[Test build #124600 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124600/testReport)** for PR 28901 at commit [`62e8ac9`](https://github.com/apache/spark/commit/62e8ac9d2b61c0aee684133418fec1bfae164930).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647945906
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654617960
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650779166
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] kevinjmh commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
kevinjmh commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-662802864
how about to use `CACHE TABLE` command to do that?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647893150
**[Test build #124382 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124382/testReport)** for PR 28901 at commit [`f3f4e15`](https://github.com/apache/spark/commit/f3f4e15cfff650d55dec50b77e52b2a7d2142014).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin edited a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin edited a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654224633
> If I write the output to a temp location and then create a temp view to read from this temp location, ...
Ah, I knew your meaning now, using `CREATE TEMP VIEW USING` command. I think it's still tricky. First, platform needs to provide the "temp" location to storage users' data which are able to clean all untracked files. It's hard to do that. Second, "CREATE TEMP VIEW USING" command lack the view definition. In most cases, immutable `temporary table` is enough. After the temporary created with `AS SELECT ...`, less insert/overwrite operations will perform on it. `CREATE TEMP VIEW USING` command without `AS SELECT`, most practises seem to `insert/overwrite` on it. What's more, the storage folder permission is still a problem in a ACL enabled platform (even Spark code didn't import ACL, many companies build their platforms similar with Databricks Runtime' ACL), and analysts are always not care about 'PATH'.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648609504
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647945315
**[Test build #124394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124394/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648537549
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648545396
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin edited a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin edited a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654224633
> If I write the output to a temp location and then create a temp view to read from this temp location, ...
Ah, I knew your meaning now, using `CREATE TEMP VIEW USING` command. I think it's still tricky. First, platform needs to provide the "temp" location to storage users' data which are able to clean all untracked files. It's hard to do that. Second, "CREATE TEMP VIEW USING" command lack the view definition. In most cases, immutable `temporary table` is enough. After the temporary created with `AS SELECT ...`, less insert/overwrite operations will perform on it. `CREATE TEMP VIEW USING` command without `AS SELECT`, a good practise seems to `insert/overwrite` it then. What's more, the storage folder permission is still a problem in a ACL enabled platform (even Spark code didn't import ACL, many companies build their platforms similar with Databricks Runtime' ACL)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648049521
**[Test build #124400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124400/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647965443
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650728623
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124600/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654644075
**[Test build #125178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125178/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647946372
**[Test build #124394 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124394/testReport)** for PR 28901 at commit [`fa1a84a`](https://github.com/apache/spark/commit/fa1a84a8fd5964d91a8b43bcb75609af43553f61).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647952602
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124397/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648050500
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648545396
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648549579
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124447/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650708232
**[Test build #124594 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124594/testReport)** for PR 28901 at commit [`6d3274f`](https://github.com/apache/spark/commit/6d3274f08c1c81262c8b0c21aa133a04e31c6796).
* This patch **fails due to an unknown error code, -9**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654676289
**[Test build #125195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125195/testReport)** for PR 28901 at commit [`9b11aac`](https://github.com/apache/spark/commit/9b11aace28be8169e8eff1ce61810bc8250fc37d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647946382
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648548682
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650747727
**[Test build #124603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124603/testReport)** for PR 28901 at commit [`62e8ac9`](https://github.com/apache/spark/commit/62e8ac9d2b61c0aee684133418fec1bfae164930).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-648545400
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124446/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650700128
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647895685
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654617960
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LantaoJin commented on pull request #28901: [WIP][SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
LantaoJin commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-650698393
@gatorsmile Yes. Just like Hive temporary table or Teradata volatile table. We are migrating our Spark to v3.0. This is one of inside features which had widely used in our prodution.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-654764058
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28901: [SPARK-32064][SQL] Supporting create temporary table
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28901:
URL: https://github.com/apache/spark/pull/28901#issuecomment-647895685
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org