You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/11/19 08:58:26 UTC
[GitHub] [spark] Peng-Lei opened a new pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Peng-Lei opened a new pull request #34666:
URL: https://github.com/apache/spark/pull/34666
### What changes were proposed in this pull request?
1. Migrate `SHOW TBLPROPERTIES` to use V2 command by default
2. Sort the result of V1 command of `SHOW TBLPROPERTIES`
3. handle the `output == 1` case for V2 command to match V1 behavior
### Why are the changes needed?
It's been a while since we introduced the v2 commands, and it seems reasonable to use v2 commands by default even for the session catalog, with a legacy config to fall back to the v1 commands.
### Does this PR introduce _any_ user-facing change?
For use V2 command:
if LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA == true
change output `Row("key", "value")` => `Row("value")`
At now it is not supported for view that `SHOW TBLPROPERTIES` command by default.
we can set `LEGACY_USE_V1_COMMAND` to use V1 command.
### How was this patch tested?
$ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *ShowTblPropertiesSuite"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r754015958
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql
##########
@@ -1,26 +0,0 @@
--- create a table with properties
-CREATE TABLE tbl (a INT, b STRING, c INT) USING parquet
-TBLPROPERTIES('p1'='v1', 'p2'='v2');
-
-SHOW TBLPROPERTIES tbl;
-SHOW TBLPROPERTIES tbl("p1");
-SHOW TBLPROPERTIES tbl("p3");
-
-DROP TABLE tbl;
-
--- create a view with properties
-CREATE VIEW view TBLPROPERTIES('p1'='v1', 'p2'='v2') AS SELECT 1 AS c1;
Review comment:
does the `ShowTblPropertiesSuiteBase` cover view?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Peng-Lei commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
Peng-Lei commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975039183
> Can you check test failures?
@imback82 I want to delete the file [show-tblproperties.sql](https://github.com/apache/spark/blob/master/sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql) to ensure that the test cases pass. As, we have the same test case in `V1.ShowTblPropertiesSuiteBase`. Do you agree?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975181998
**[Test build #145499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145499/testReport)** for PR 34666 at commit [`6e8ac96`](https://github.com/apache/spark/commit/6e8ac96b747512d9974c541af75beda09c8506ec).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-974000042
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49923/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975631207
**[Test build #145509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145509/testReport)** for PR 34666 at commit [`fb34b48`](https://github.com/apache/spark/commit/fb34b48536d0f43a591310fe65b8348e4459c99b).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975664103
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145509/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975431880
**[Test build #145499 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145499/testReport)** for PR 34666 at commit [`6e8ac96`](https://github.com/apache/spark/commit/6e8ac96b747512d9974c541af75beda09c8506ec).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975287934
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-973998450
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145449/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975442169
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975441782
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975658458
The GA job fails `org.apache.spark.ml.source.image.ImageFileFormatSuite`
This is unrelated and passed in jenkins. I'm merging this to master, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-974000021
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49923/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Peng-Lei commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
Peng-Lei commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r753982634
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala
##########
@@ -56,7 +44,7 @@ trait ShowTblPropertiesSuiteBase extends command.ShowTblPropertiesSuiteBase
}
}
- test("SHOW TBLPROPERTIES FOR TEMPORARY IEW") {
+ testV1("SHOW TBLPROPERTIES FOR TEMPORARY IEW") {
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Peng-Lei commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
Peng-Lei commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r754023053
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql
##########
@@ -1,26 +0,0 @@
--- create a table with properties
-CREATE TABLE tbl (a INT, b STRING, c INT) USING parquet
-TBLPROPERTIES('p1'='v1', 'p2'='v2');
-
-SHOW TBLPROPERTIES tbl;
-SHOW TBLPROPERTIES tbl("p1");
-SHOW TBLPROPERTIES tbl("p3");
-
-DROP TABLE tbl;
-
--- create a view with properties
-CREATE VIEW view TBLPROPERTIES('p1'='v1', 'p2'='v2') AS SELECT 1 AS c1;
Review comment:
I think [here](https://github.com/apache/spark/blob/06586f681bb72c4ad48ef9203e290d75b2732670/sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala#L50) cover the view test. but just for v1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975181998
**[Test build #145499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145499/testReport)** for PR 34666 at commit [`6e8ac96`](https://github.com/apache/spark/commit/6e8ac96b747512d9974c541af75beda09c8506ec).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975287934
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975442133
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r754016866
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala
##########
@@ -56,7 +44,7 @@ trait ShowTblPropertiesSuiteBase extends command.ShowTblPropertiesSuiteBase
}
}
- test("SHOW TBLPROPERTIES FOR TEMPORARY IEW") {
+ testV1("SHOW TBLPROPERTIES FOR TEMPORARY VIEW") {
Review comment:
are you saying, by default `SHOW TBLPROPERTIES` can't work with temp view? That's a terrible breaking change and we should fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r754016866
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala
##########
@@ -56,7 +44,7 @@ trait ShowTblPropertiesSuiteBase extends command.ShowTblPropertiesSuiteBase
}
}
- test("SHOW TBLPROPERTIES FOR TEMPORARY IEW") {
+ testV1("SHOW TBLPROPERTIES FOR TEMPORARY VIEW") {
Review comment:
are you saying, after this PR, by default `SHOW TBLPROPERTIES` can't work with temp view? That's a terrible breaking change and we should fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r754040410
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala
##########
@@ -56,7 +44,7 @@ trait ShowTblPropertiesSuiteBase extends command.ShowTblPropertiesSuiteBase
}
}
- test("SHOW TBLPROPERTIES FOR TEMPORARY IEW") {
+ testV1("SHOW TBLPROPERTIES FOR TEMPORARY VIEW") {
Review comment:
A simple idea is to force to use v1 command if the target is a view
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975255133
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975441782
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145499/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975362023
**[Test build #145509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145509/testReport)** for PR 34666 at commit [`fb34b48`](https://github.com/apache/spark/commit/fb34b48536d0f43a591310fe65b8348e4459c99b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-973881098
**[Test build #145449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145449/testReport)** for PR 34666 at commit [`c4da96f`](https://github.com/apache/spark/commit/c4da96f46a74002ac6cede8c25787ac0b578b4b2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Peng-Lei commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
Peng-Lei commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-973895291
@imback82 @cloud-fan If you have time, Could you take a look? Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975209500
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975664103
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145509/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975404669
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-973978757
**[Test build #145449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145449/testReport)** for PR 34666 at commit [`c4da96f`](https://github.com/apache/spark/commit/c4da96f46a74002ac6cede8c25787ac0b578b4b2).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Peng-Lei commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
Peng-Lei commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r753982833
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/ShowTblPropertiesSuiteBase.scala
##########
@@ -87,4 +88,25 @@ trait ShowTblPropertiesSuiteBase extends QueryTest with DDLCommandTestUtils {
assert(res.head.getString(1).contains(s"does not have property: $nonExistingKey"))
}
}
+
+ test("KEEP THE LEGACY OUTPUT SCHEMA") {
+ Seq(true, false).foreach { keepLegacySchema =>
+ withSQLConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA.key -> keepLegacySchema.toString) {
+ withNamespaceAndTable("ns1", "tbl") { tbl =>
+ spark.sql(s"CREATE TABLE $tbl (id bigint, data string) $defaultUsing " +
+ s"TBLPROPERTIES ('user'='spark', 'status'='new')")
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Peng-Lei commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
Peng-Lei commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r754023254
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala
##########
@@ -56,7 +44,7 @@ trait ShowTblPropertiesSuiteBase extends command.ShowTblPropertiesSuiteBase
}
}
- test("SHOW TBLPROPERTIES FOR TEMPORARY IEW") {
+ testV1("SHOW TBLPROPERTIES FOR TEMPORARY VIEW") {
Review comment:
ok. I try do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-973962547
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49923/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-973881098
**[Test build #145449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145449/testReport)** for PR 34666 at commit [`c4da96f`](https://github.com/apache/spark/commit/c4da96f46a74002ac6cede8c25787ac0b578b4b2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #34666:
URL: https://github.com/apache/spark/pull/34666
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-975362023
**[Test build #145509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145509/testReport)** for PR 34666 at commit [`fb34b48`](https://github.com/apache/spark/commit/fb34b48536d0f43a591310fe65b8348e4459c99b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
imback82 commented on a change in pull request #34666:
URL: https://github.com/apache/spark/pull/34666#discussion_r753622693
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/ShowTblPropertiesSuiteBase.scala
##########
@@ -87,4 +88,25 @@ trait ShowTblPropertiesSuiteBase extends QueryTest with DDLCommandTestUtils {
assert(res.head.getString(1).contains(s"does not have property: $nonExistingKey"))
}
}
+
+ test("KEEP THE LEGACY OUTPUT SCHEMA") {
+ Seq(true, false).foreach { keepLegacySchema =>
+ withSQLConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA.key -> keepLegacySchema.toString) {
+ withNamespaceAndTable("ns1", "tbl") { tbl =>
+ spark.sql(s"CREATE TABLE $tbl (id bigint, data string) $defaultUsing " +
+ s"TBLPROPERTIES ('user'='spark', 'status'='new')")
Review comment:
nit: `s` not needed.
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -401,15 +401,9 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
throw QueryCompilationErrors.externalCatalogNotSupportShowViewsError(resolved)
}
- case s @ ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey, output) =>
- val newOutput =
- if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA) && propertyKey.isDefined) {
- assert(output.length == 2)
- output.tail
- } else {
- output
- }
- ShowTablePropertiesCommand(ident.asTableIdentifier, propertyKey, newOutput)
+ case s @ ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey, output)
+ if conf.useV1Command =>
+ ShowTablePropertiesCommand(ident.asTableIdentifier, propertyKey, output)
Review comment:
nit:
```suggestion
if conf.useV1Command =>
ShowTablePropertiesCommand(ident.asTableIdentifier, propertyKey, output)
```
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala
##########
@@ -56,7 +44,7 @@ trait ShowTblPropertiesSuiteBase extends command.ShowTblPropertiesSuiteBase
}
}
- test("SHOW TBLPROPERTIES FOR TEMPORARY IEW") {
+ testV1("SHOW TBLPROPERTIES FOR TEMPORARY IEW") {
Review comment:
nit: IEW => VIEW
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-973998450
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145449/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34666:
URL: https://github.com/apache/spark/pull/34666#issuecomment-974000042
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49923/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org