You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/03/11 03:18:09 UTC
[GitHub] [spark] yaooqinn opened a new pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
yaooqinn opened a new pull request #31804:
URL: https://github.com/apache/spark/pull/31804
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
SHOW TABLES has an output column `isTemporary` which only indicates if a view is a local temp view that is not enough for users if they want a pipeline of Spark commands, such as SHOW TABLES foreach { if isView DROP VIEW else DROP TABLE}
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
distinguish view and tables
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
yes, show tables and show table extended will have one new column called `isView` at the end
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
ut modified
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807997064
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41129/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807969381
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796951283
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40561/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592081649
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -514,7 +514,8 @@ object ShowTables {
def getOutputAttrs: Seq[Attribute] = Seq(
AttributeReference("namespace", StringType, nullable = false)(),
AttributeReference("tableName", StringType, nullable = false)(),
- AttributeReference("isTemporary", BooleanType, nullable = false)())
+ AttributeReference("isTemporary", BooleanType, nullable = false)(),
+ AttributeReference("isView", BooleanType, nullable = false)())
Review comment:
JDBC protocol also use table type, right?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592068396
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -366,13 +366,13 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
partitionSpec @ (None | Some(UnresolvedPartitionSpec(_, _))),
output) =>
val newOutput = if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
- assert(output.length == 4)
- output.head.withName("database") +: output.tail
+ assert(output.length == 5)
Review comment:
Introducing a new `legacy` conf for this behavior change seems kind of trivial and might bring cognition burdens for users. So the config is reused for now and the doc will be updated if it is the right way to go
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797273140
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40576/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-808110502
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136545/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797611691
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40596/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807950640
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136544/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797523002
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136002/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796847294
**[Test build #135975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135975/testReport)** for PR 31804 at commit [`6bc1614`](https://github.com/apache/spark/commit/6bc16144d2ec465ba169a208c86f015eedc88f04).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592050801
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tables-legacy.sql
##########
@@ -0,0 +1,2 @@
+--SET spark.sql.legacy.keepCommandOutputSchema=true
Review comment:
This might be useful to verify the legacy schema
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796982349
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40561/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796847294
**[Test build #135975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135975/testReport)** for PR 31804 at commit [`6bc1614`](https://github.com/apache/spark/commit/6bc16144d2ec465ba169a208c86f015eedc88f04).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797523002
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136002/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807878549
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810075013
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41286/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797292989
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135992/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592065605
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -533,7 +534,8 @@ object ShowTableExtended {
AttributeReference("namespace", StringType, nullable = false)(),
AttributeReference("tableName", StringType, nullable = false)(),
AttributeReference("isTemporary", BooleanType, nullable = false)(),
- AttributeReference("information", StringType, nullable = false)())
+ AttributeReference("information", StringType, nullable = false)(),
+ AttributeReference("isView", BooleanType, nullable = false)())
Review comment:
yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796462969
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797292989
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135992/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796982349
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40561/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797359483
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40580/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797611691
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40596/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797551323
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40589/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796457946
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40543/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810069519
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41286/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593782353
##########
File path: sql/core/src/test/resources/log4j.properties
##########
@@ -22,7 +22,7 @@ log4j.rootLogger=INFO, CA, FA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n
-log4j.appender.CA.Threshold = WARN
+log4j.appender.CA.Threshold = FATAL
Review comment:
But the current threshold (WARN) can help catch a bug like the following early: https://github.com/apache/spark/pull/31273#issuecomment-785854571
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810252384
**[Test build #136705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136705/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797176421
> Could you add some query examples for this behaviour change in the PR description? Also, I think we need to update the migration guide, too.
Thanks, @maropu, your suggestions sound good~
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810740574
cc @cloud-fan @HyukjinKwon PTAL, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822340178
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137568/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810074980
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41286/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797395459
**[Test build #136002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136002/testReport)** for PR 31804 at commit [`5e9dbcc`](https://github.com/apache/spark/commit/5e9dbccef8ec865b3d1caad7da4fdcedb21e9943).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822166853
**[Test build #137568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137568/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592063771
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -533,7 +534,8 @@ object ShowTableExtended {
AttributeReference("namespace", StringType, nullable = false)(),
AttributeReference("tableName", StringType, nullable = false)(),
AttributeReference("isTemporary", BooleanType, nullable = false)(),
- AttributeReference("information", StringType, nullable = false)())
+ AttributeReference("information", StringType, nullable = false)(),
+ AttributeReference("isView", BooleanType, nullable = false)())
Review comment:
Is it added at the last to reduce the breaking change effect?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592072632
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -514,7 +514,8 @@ object ShowTables {
def getOutputAttrs: Seq[Attribute] = Seq(
AttributeReference("namespace", StringType, nullable = false)(),
AttributeReference("tableName", StringType, nullable = false)(),
- AttributeReference("isTemporary", BooleanType, nullable = false)())
+ AttributeReference("isTemporary", BooleanType, nullable = false)(),
+ AttributeReference("isView", BooleanType, nullable = false)())
Review comment:
A new column `isView: Boolean` is more efficient, but I'm wondering if a new column `tableType: String` is more user-friendly. The value can be `TABLE` or `VIEW`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593906834
##########
File path: sql/core/src/test/resources/log4j.properties
##########
@@ -22,7 +22,7 @@ log4j.rootLogger=INFO, CA, FA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n
-log4j.appender.CA.Threshold = WARN
+log4j.appender.CA.Threshold = FATAL
Review comment:
> But the current threshold (WARN) can help catch a bug like the following early: [#31273 (comment)](https://github.com/apache/spark/pull/31273#issuecomment-785854571)
These warning messages still can be found in the `unit-tests.log`s. I don't see much difference as a warning message can still be simply ignored.
But when we encounter test failures but got omitted by the CI, it is hard for us to locate to them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807884538
**[Test build #136543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136543/testReport)** for PR 31804 at commit [`bee8cbe`](https://github.com/apache/spark/commit/bee8cbee33ceeca81a25d746a63720eb7fe78cd7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807969382
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r601965623
##########
File path: sql/core/src/test/resources/log4j.properties
##########
@@ -22,7 +22,7 @@ log4j.rootLogger=INFO, CA, FA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n
-log4j.appender.CA.Threshold = WARN
+log4j.appender.CA.Threshold = FATAL
Review comment:
I think it's okay as long as `unit-tests.log` contains.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592070614
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -843,11 +843,22 @@ case class ShowTablesCommand(
val database = tableIdent.database.getOrElse("")
val tableName = tableIdent.table
val isTemp = catalog.isTempView(tableIdent)
+ val catalogTable = catalog.getTempViewOrPermanentTableMetadata(tableIdent)
+ val isView = catalogTable.tableType == CatalogTableType.VIEW
if (isExtended) {
- val information = catalog.getTempViewOrPermanentTableMetadata(tableIdent).simpleString
- Row(database, tableName, isTemp, s"$information\n")
+ val information = catalogTable.simpleString
+ if (output.size == 5) {
+ Row(database, tableName, isTemp, s"$information\n", isView)
+ } else {
+ Row(database, tableName, isTemp, s"$information\n")
+ }
} else {
- Row(database, tableName, isTemp)
+ if (output.size == 4) {
+ Row(database, tableName, isTemp, isView)
+ } else {
+ Row(database, tableName, isTemp)
Review comment:
yes, the new `show-tables-legacy.sql` will import the corresponding tests to cover. I can add some cases in `v1.ShowTablesSuite` if `show-tables-legacy.sql` is unintuitive
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796461652
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40543/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797359483
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40580/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807910778
**[Test build #136544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136544/testReport)** for PR 31804 at commit [`eaba257`](https://github.com/apache/spark/commit/eaba25705b82f5462246c8b6fbb89321f7e43ee2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-808110502
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136545/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796915167
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135977/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592075370
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -514,7 +514,8 @@ object ShowTables {
def getOutputAttrs: Seq[Attribute] = Seq(
AttributeReference("namespace", StringType, nullable = false)(),
AttributeReference("tableName", StringType, nullable = false)(),
- AttributeReference("isTemporary", BooleanType, nullable = false)())
+ AttributeReference("isTemporary", BooleanType, nullable = false)(),
+ AttributeReference("isView", BooleanType, nullable = false)())
Review comment:
Usually, most modern databases store `tableType` as a string column in `INFORMATION_SCHEMA.TABLES`.
On the reading side, they also return a string value for it.
FYI, https://docs.google.com/spreadsheets/d/1LeHYbGCDjgr-rYwQMlHBhJbeDxjOVuUatozgvKdxq6Y/edit#gid=0
Besides, as `SHOW TABLES` is not ANSI standard, so it might be good for us to follow the JDBC standard. Then we can make our command and JDBC meta operation consistent
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593634764
##########
File path: sql/core/src/test/resources/log4j.properties
##########
@@ -22,7 +22,7 @@ log4j.rootLogger=INFO, CA, FA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n
-log4j.appender.CA.Threshold = WARN
+log4j.appender.CA.Threshold = FATAL
Review comment:
If so, Is it better to backport this change into the previous branches? cc: @HyukjinKwon @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796968019
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40561/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796860722
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135975/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593366325
##########
File path: sql/core/src/test/resources/log4j.properties
##########
@@ -22,7 +22,7 @@ log4j.rootLogger=INFO, CA, FA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n
-log4j.appender.CA.Threshold = WARN
+log4j.appender.CA.Threshold = FATAL
Review comment:
oversized log for GA console output cause truncation fo useful test errors, andthe unit-tests.log is enough
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807937788
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41128/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796424679
**[Test build #135958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135958/testReport)** for PR 31804 at commit [`2546812`](https://github.com/apache/spark/commit/25468129f57655d2bf4166efdf89714f969cbd76).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807884538
**[Test build #136543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136543/testReport)** for PR 31804 at commit [`bee8cbe`](https://github.com/apache/spark/commit/bee8cbee33ceeca81a25d746a63720eb7fe78cd7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797395459
**[Test build #136002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136002/testReport)** for PR 31804 at commit [`5e9dbcc`](https://github.com/apache/spark/commit/5e9dbccef8ec865b3d1caad7da4fdcedb21e9943).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592087833
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -514,7 +514,8 @@ object ShowTables {
def getOutputAttrs: Seq[Attribute] = Seq(
AttributeReference("namespace", StringType, nullable = false)(),
AttributeReference("tableName", StringType, nullable = false)(),
- AttributeReference("isTemporary", BooleanType, nullable = false)())
+ AttributeReference("isTemporary", BooleanType, nullable = false)(),
+ AttributeReference("isView", BooleanType, nullable = false)())
Review comment:
Yes, another benefit for using string is if we decide to subdivide tables to something like `SYSTEM TABLE`/`BASE TABLE`, `xxx TABLE` e.t.c, we won't break then
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592072172
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -366,13 +366,13 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
partitionSpec @ (None | Some(UnresolvedPartitionSpec(_, _))),
output) =>
val newOutput = if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
- assert(output.length == 4)
- output.head.withName("database") +: output.tail
+ assert(output.length == 5)
Review comment:
> this should be output.length == 4
indeed, this is true. `output.head.withName("database") +: output.slice(1, 4)` will cut the `isView` off
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593753415
##########
File path: sql/core/src/test/resources/log4j.properties
##########
@@ -22,7 +22,7 @@ log4j.rootLogger=INFO, CA, FA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n
-log4j.appender.CA.Threshold = WARN
+log4j.appender.CA.Threshold = FATAL
Review comment:
yeah, I guess so. we can mute console output for most modules. we call `o.s.Assertions.intercept` frequently which produces a lot of unnecessary error logs.
When we mute them, the error stacktraces for failed tests can still be kept.
I can make a separate PR to fix it if it makes sense to the CCers.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797152384
Could you add some query examples for this behaviour change in the PR description? Also, I think we need to update the migration guide, too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796464724
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135958/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #31804:
URL: https://github.com/apache/spark/pull/31804
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797404623
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40586/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822591847
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137604/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797505194
**[Test build #136002 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136002/testReport)** for PR 31804 at commit [`5e9dbcc`](https://github.com/apache/spark/commit/5e9dbccef8ec865b3d1caad7da4fdcedb21e9943).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807948263
**[Test build #136545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136545/testReport)** for PR 31804 at commit [`f50d87b`](https://github.com/apache/spark/commit/f50d87b58e3b01b5f9451ce523080b7f58e0d6e3).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592069590
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -366,13 +366,13 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
partitionSpec @ (None | Some(UnresolvedPartitionSpec(_, _))),
output) =>
val newOutput = if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
- assert(output.length == 4)
- output.head.withName("database") +: output.tail
+ assert(output.length == 5)
Review comment:
This is not reused technically. If we reuse the existing conf, this should be `output.length == 4` because it disable this PR and the previous commit simultaneously.
> So the config is reused for now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807944124
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41128/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797332066
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135996/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797573140
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796458548
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40542/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807948263
**[Test build #136545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136545/testReport)** for PR 31804 at commit [`f50d87b`](https://github.com/apache/spark/commit/f50d87b58e3b01b5f9451ce523080b7f58e0d6e3).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797612823
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136012/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807954099
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41127/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-808108299
**[Test build #136545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136545/testReport)** for PR 31804 at commit [`f50d87b`](https://github.com/apache/spark/commit/f50d87b58e3b01b5f9451ce523080b7f58e0d6e3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810254531
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136705/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796860722
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135975/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797404623
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40586/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797531345
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40589/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592070973
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tables-legacy.sql
##########
@@ -0,0 +1,2 @@
+--SET spark.sql.legacy.keepCommandOutputSchema=true
Review comment:
the code change is related, `spark.sql.legacy.keepCommandOutputSchema=true` will omit the `isView` column
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592066222
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -843,11 +843,22 @@ case class ShowTablesCommand(
val database = tableIdent.database.getOrElse("")
val tableName = tableIdent.table
val isTemp = catalog.isTempView(tableIdent)
+ val catalogTable = catalog.getTempViewOrPermanentTableMetadata(tableIdent)
+ val isView = catalogTable.tableType == CatalogTableType.VIEW
if (isExtended) {
- val information = catalog.getTempViewOrPermanentTableMetadata(tableIdent).simpleString
- Row(database, tableName, isTemp, s"$information\n")
+ val information = catalogTable.simpleString
+ if (output.size == 5) {
+ Row(database, tableName, isTemp, s"$information\n", isView)
+ } else {
+ Row(database, tableName, isTemp, s"$information\n")
+ }
} else {
- Row(database, tableName, isTemp)
+ if (output.size == 4) {
+ Row(database, tableName, isTemp, isView)
+ } else {
+ Row(database, tableName, isTemp)
Review comment:
Do we still have a test coverage for this line?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822340178
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137568/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822340496
**[Test build #137604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137604/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797332066
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135996/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796464600
**[Test build #135958 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135958/testReport)** for PR 31804 at commit [`2546812`](https://github.com/apache/spark/commit/25468129f57655d2bf4166efdf89714f969cbd76).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797662928
**[Test build #136005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136005/testReport)** for PR 31804 at commit [`bee8cbe`](https://github.com/apache/spark/commit/bee8cbee33ceeca81a25d746a63720eb7fe78cd7).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822340496
**[Test build #137604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137604/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593636880
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -510,11 +510,21 @@ case class ShowTables(
override def children: Seq[LogicalPlan] = Seq(namespace)
}
-object ShowTables {
+trait ShowTablesLegacyHelper {
+ def getOutputAttrs: Seq[Attribute]
Review comment:
Is this a good idea that the code to handle this legacy behaviour depends on `trait`? If we remove this legacy behaviour in far future and we remove this `trait`, the change can lead to binary-incompatibility?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796853263
**[Test build #135975 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135975/testReport)** for PR 31804 at commit [`6bc1614`](https://github.com/apache/spark/commit/6bc16144d2ec465ba169a208c86f015eedc88f04).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `trait ShowTablesLegacyHelper `
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797672574
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136005/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r601967696
##########
File path: sql/core/src/test/resources/log4j.properties
##########
@@ -22,7 +22,7 @@ log4j.rootLogger=INFO, CA, FA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n
-log4j.appender.CA.Threshold = WARN
+log4j.appender.CA.Threshold = FATAL
Review comment:
Thanks, I'll send a PR then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592065425
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tables-legacy.sql
##########
@@ -0,0 +1,2 @@
+--SET spark.sql.legacy.keepCommandOutputSchema=true
Review comment:
Is it relevant to this PR's `isView` column addition?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807927940
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41128/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810034572
**[Test build #136705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136705/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796443288
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40542/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r603859991
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -510,11 +510,21 @@ case class ShowTables(
override def children: Seq[LogicalPlan] = Seq(namespace)
}
-object ShowTables {
+trait ShowTablesLegacyHelper {
+ def getOutputAttrs: Seq[Attribute]
Review comment:
(Sorry for my late reply...) How about simply inlining `getLegacyOutputAttrs`? It seems there are the only two places where `getLegacyOutputAttrs` is used, so I'm not sure that we need this trait.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822591847
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137604/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796462969
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592064395
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -366,13 +366,13 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
partitionSpec @ (None | Some(UnresolvedPartitionSpec(_, _))),
output) =>
val newOutput = if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
- assert(output.length == 4)
- output.head.withName("database") +: output.tail
+ assert(output.length == 5)
+ output.head.withName("database") +: output.slice(1, 4)
} else {
output
}
val tablePartitionSpec = partitionSpec.map(_.asInstanceOf[UnresolvedPartitionSpec].spec)
- ShowTablesCommand(Some(db), Some(pattern), newOutput, true, tablePartitionSpec)
+ ShowTablesCommand(Some(db), Some(pattern), newOutput, isExtended = true, tablePartitionSpec)
Review comment:
I know this is a recommended style, but if you don't mind, shall we avoid this change in this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822335111
**[Test build #137568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137568/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810254531
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136705/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592064976
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -366,13 +366,13 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
partitionSpec @ (None | Some(UnresolvedPartitionSpec(_, _))),
output) =>
val newOutput = if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
- assert(output.length == 4)
- output.head.withName("database") +: output.tail
+ assert(output.length == 5)
+ output.head.withName("database") +: output.slice(1, 4)
} else {
output
}
val tablePartitionSpec = partitionSpec.map(_.asInstanceOf[UnresolvedPartitionSpec].spec)
- ShowTablesCommand(Some(db), Some(pattern), newOutput, true, tablePartitionSpec)
+ ShowTablesCommand(Some(db), Some(pattern), newOutput, isExtended = true, tablePartitionSpec)
Review comment:
make sense
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797612823
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136012/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807998713
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41129/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807971257
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41129/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822166853
**[Test build #137568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137568/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810034572
**[Test build #136705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136705/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807925937
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41127/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807998713
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41129/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807944124
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41128/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593753415
##########
File path: sql/core/src/test/resources/log4j.properties
##########
@@ -22,7 +22,7 @@ log4j.rootLogger=INFO, CA, FA
log4j.appender.CA=org.apache.log4j.ConsoleAppender
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%d{HH:mm:ss.SSS} %p %c: %m%n
-log4j.appender.CA.Threshold = WARN
+log4j.appender.CA.Threshold = FATAL
Review comment:
yeah, I guess so. we can mute console output for most modules. we call `o.s.Assertions.intercept` frequently which produces a lot of unnecessary error logs.
When we mute them, the error stacktraces for failed tests can still be kept.
I can make a separate PR to fix it if it makes senses to the CCers too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807950460
**[Test build #136544 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136544/testReport)** for PR 31804 at commit [`eaba257`](https://github.com/apache/spark/commit/eaba25705b82f5462246c8b6fbb89321f7e43ee2).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796424679
**[Test build #135958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135958/testReport)** for PR 31804 at commit [`2546812`](https://github.com/apache/spark/commit/25468129f57655d2bf4166efdf89714f969cbd76).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797561486
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40589/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592071270
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tables-legacy.sql
##########
@@ -0,0 +1,2 @@
+--SET spark.sql.legacy.keepCommandOutputSchema=true
Review comment:
For instance,
https://github.com/apache/spark/pull/31804/files#diff-a854c4d2d3a463feef2307548a917bae452850a16de730f8a14f84a4eb79a16fR64
https://github.com/apache/spark/pull/31804/files#diff-b6f30759017988fd0963ce840918f541caf610a1793fa73f17e61b03a2acb797R64
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593636880
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -510,11 +510,21 @@ case class ShowTables(
override def children: Seq[LogicalPlan] = Seq(namespace)
}
-object ShowTables {
+trait ShowTablesLegacyHelper {
+ def getOutputAttrs: Seq[Attribute]
Review comment:
Is this a good idea that the code to handle this legacy behaviour depends on `tirait`? If we remove this legacy behaviour in far future and we remove this trait, the change can lead to binary-incompatibility?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807910778
**[Test build #136544 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136544/testReport)** for PR 31804 at commit [`eaba257`](https://github.com/apache/spark/commit/eaba25705b82f5462246c8b6fbb89321f7e43ee2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r593752852
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala
##########
@@ -510,11 +510,21 @@ case class ShowTables(
override def children: Seq[LogicalPlan] = Seq(namespace)
}
-object ShowTables {
+trait ShowTablesLegacyHelper {
+ def getOutputAttrs: Seq[Attribute]
Review comment:
The callers are bundled in the catalyst module, I guess it's safe?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797494362
**[Test build #136005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136005/testReport)** for PR 31804 at commit [`bee8cbe`](https://github.com/apache/spark/commit/bee8cbee33ceeca81a25d746a63720eb7fe78cd7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31804:
URL: https://github.com/apache/spark/pull/31804#discussion_r592066062
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -366,13 +366,13 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
partitionSpec @ (None | Some(UnresolvedPartitionSpec(_, _))),
output) =>
val newOutput = if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
- assert(output.length == 4)
- output.head.withName("database") +: output.tail
+ assert(output.length == 5)
Review comment:
Well, this seems inconsistent with the doc. The current document means `spark.sql.legacy.keepCommandOutputSchema` means the 3.1 or earlier schema, doesn't it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796444436
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135959/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
yaooqinn commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796428494
cc @cloud-fan @dongjoon-hyun @HyukjinKwon @maropu thanks for the review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797494362
**[Test build #136005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136005/testReport)** for PR 31804 at commit [`bee8cbe`](https://github.com/apache/spark/commit/bee8cbee33ceeca81a25d746a63720eb7fe78cd7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-888700037
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807957257
**[Test build #136543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136543/testReport)** for PR 31804 at commit [`bee8cbe`](https://github.com/apache/spark/commit/bee8cbee33ceeca81a25d746a63720eb7fe78cd7).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796915167
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135977/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-807950640
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136544/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-822590146
**[Test build #137604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137604/testReport)** for PR 31804 at commit [`3ef5b5f`](https://github.com/apache/spark/commit/3ef5b5f414b68ea149877eaa26618e9787252004).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797273140
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40576/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-810075013
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41286/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796444436
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135959/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796872471
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40559/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-797561486
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40589/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31804: [SPARK-34710][SQL] Add isView for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796464724
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135958/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31804: [SPARK-34710][SQL] Add tableType column for SHOW TABLES to distinguish view and tables
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31804:
URL: https://github.com/apache/spark/pull/31804#issuecomment-796872471
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40559/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org