You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/28 06:24:41 UTC
[GitHub] [spark] AngersZhuuuu opened a new pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
AngersZhuuuu opened a new pull request #31378:
URL: https://github.com/apache/spark/pull/31378
### What changes were proposed in this pull request?
Passing around the output attributes should have more benefits like keeping the expr ID unchanged to avoid bugs when we apply more operators above the command output dataframe.
This PR keep SHOW TBLPROPERTIES command's output attribute exprId unchanged.
### Why are the changes needed?
Keep SHOW PARTITIONS command's output attribute exprid unchanged.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Added UT
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774594283
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39553/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774633380
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134970/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771889320
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134783/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769614748
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134632/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r570985029
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -490,8 +490,8 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
throw QueryCompilationErrors.externalCatalogNotSupportShowViewsError(resolved)
}
- case ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey) =>
- ShowTablePropertiesCommand(ident.asTableIdentifier, propertyKey)
+ case s @ ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey) =>
Review comment:
let's add the `output` field to `ShowTableProperties` as well, following https://github.com/apache/spark/pull/31245/files#diff-594addaa7ed3dd43fd0dad5fa81ade5ec2e570adf7c982c0570e3358a08a8e0aR490
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775184044
**[Test build #135032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135032/testReport)** for PR 31378 at commit [`ef6b45c`](https://github.com/apache/spark/commit/ef6b45cca32c9f405f364c942476eaa654f903cd).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772492192
**[Test build #134820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134820/testReport)** for PR 31378 at commit [`9f850c6`](https://github.com/apache/spark/commit/9f850c6926fad397fdafa545df03b299c1a41f7f).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774586327
**[Test build #134970 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134970/testReport)** for PR 31378 at commit [`9475d66`](https://github.com/apache/spark/commit/9475d665feba1cde63a86536badb5212efe642e2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772322076
**[Test build #134820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134820/testReport)** for PR 31378 at commit [`9f850c6`](https://github.com/apache/spark/commit/9f850c6926fad397fdafa545df03b299c1a41f7f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568382649
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> Changing the output schema looks fine to me. Could you describe this change in the PR description?
Yea, updated, Do we need to change migration guide too?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771651447
**[Test build #134783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134783/testReport)** for PR 31378 at commit [`8552df4`](https://github.com/apache/spark/commit/8552df4a518e67665d83d3b3578683f3140bb595).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774632174
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134973/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768887314
**[Test build #134590 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134590/testReport)** for PR 31378 at commit [`af15746`](https://github.com/apache/spark/commit/af157469aa0138c484382278c5b03563b6b97220).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class ShowTablePropertiesCommand(`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771711068
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39370/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768897473
**[Test build #134609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134609/testReport)** for PR 31378 at commit [`351cba7`](https://github.com/apache/spark/commit/351cba7551484406908ac00756d797cabf92aa71).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772493396
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567844130
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -895,16 +895,10 @@ case class ShowTablesCommand(
* SHOW TBLPROPERTIES table_name[('propertyKey')];
* }}}
*/
-case class ShowTablePropertiesCommand(table: TableIdentifier, propertyKey: Option[String])
- extends RunnableCommand {
-
- override val output: Seq[Attribute] = {
- val schema = AttributeReference("value", StringType, nullable = false)() :: Nil
- propertyKey match {
- case None => AttributeReference("key", StringType, nullable = false)() :: schema
- case _ => schema
- }
- }
+case class ShowTablePropertiesCommand(
+ table: TableIdentifier,
+ propertyKey: Option[String],
+ override val output: Seq[Attribute]) extends RunnableCommand {
Review comment:
Same change as last PR https://github.com/apache/spark/pull/31341/files
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771550067
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r571998057
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql
##########
@@ -6,6 +6,13 @@ SHOW TBLPROPERTIES tbl;
SHOW TBLPROPERTIES tbl("p1");
SHOW TBLPROPERTIES tbl("p3");
+set spark.sql.legacy.keepCommandOutputSchema=true;
Review comment:
> Can we follow other PRs and test the config in the scala test?
Updated, should we merge `test("SPARK-34240 Unify output of SHOW TBLPROPERTIES and pass output attributes properly") ` and `test("show tblproperties for hive table") `
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -481,8 +481,14 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
throw QueryCompilationErrors.externalCatalogNotSupportShowViewsError(resolved)
}
- case ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey) =>
- ShowTablePropertiesCommand(ident.asTableIdentifier, propertyKey)
+ case s @ ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey, output) =>
+ val newOutput = if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
+ assert(output.length == 2)
+ output.tail
Review comment:
> we should only do it if key is specified.
Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r569239780
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the table property `key`. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when you specify the table property `key`.
Review comment:
"In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only shows ..."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775157465
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39610/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772351760
> Why are the changes needed?
> Keep SHOW PARTITIONS command's output attribute exprId unchanged.
SHOW PARTITIONS? SHOW TBLPROPERTIES?
Besides, seems this PR does more in keep the commend output consistent. I think this is one of reason why the change needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769614050
**[Test build #134632 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134632/testReport)** for PR 31378 at commit [`1036b2d`](https://github.com/apache/spark/commit/1036b2d732fdf173f905c117dc7f89d2c2572769).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772358817
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769569104
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39220/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774590677
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39553/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775180757
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39610/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567894564
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
then shall we change the schema of the v2 command?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772383181
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39407/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772354994
> Please also update the PR title too. Currently it looks not very meaningful.
How about current?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775222456
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39616/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771689856
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39370/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771840438
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134780/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768838711
**[Test build #134590 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134590/testReport)** for PR 31378 at commit [`af15746`](https://github.com/apache/spark/commit/af157469aa0138c484382278c5b03563b6b97220).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775139197
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39610/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772383181
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39407/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r571983429
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql
##########
@@ -6,6 +6,13 @@ SHOW TBLPROPERTIES tbl;
SHOW TBLPROPERTIES tbl("p1");
SHOW TBLPROPERTIES tbl("p3");
+set spark.sql.legacy.keepCommandOutputSchema=true;
Review comment:
Can we follow other PRs and test the config in the scala test?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771817567
**[Test build #134780 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134780/testReport)** for PR 31378 at commit [`aa348e1`](https://github.com/apache/spark/commit/aa348e19611d4d1e72ffe5978500cea9a310bb70).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771734982
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39370/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774586327
**[Test build #134970 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134970/testReport)** for PR 31378 at commit [`9475d66`](https://github.com/apache/spark/commit/9475d665feba1cde63a86536badb5212efe642e2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775293815
**[Test build #135027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135027/testReport)** for PR 31378 at commit [`728fca4`](https://github.com/apache/spark/commit/728fca4979bc7251168e8a68d85576fd97cc48fe).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567846183
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> is it a reasonable behavior change?
To be honest I am confused that why in `ShowTablePropertiesCommand` we don't show `key` if not specified
but we will show always show `key` in `ShowTableProperties`
Keep consistent I think is necessary.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771487869
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775398487
**[Test build #135037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135037/testReport)** for PR 31378 at commit [`cc9d8ac`](https://github.com/apache/spark/commit/cc9d8ac1105a336997384eab475fc82458d92b01).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774584708
**[Test build #134973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134973/testReport)** for PR 31378 at commit [`b0cf93f`](https://github.com/apache/spark/commit/b0cf93f830fa49767aa0ace69ef14e816b8080b5).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775419237
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135037/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769016259
**[Test build #134609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134609/testReport)** for PR 31378 at commit [`351cba7`](https://github.com/apache/spark/commit/351cba7551484406908ac00756d797cabf92aa71).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769534379
**[Test build #134632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134632/testReport)** for PR 31378 at commit [`1036b2d`](https://github.com/apache/spark/commit/1036b2d732fdf173f905c117dc7f89d2c2572769).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567899325
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
But seems changed in hive current master branch https://github.com/apache/hive/blob/95c7d361549272c52dfcf637eb2a0360b79b9815/ql/src/java/org/apache/hadoop/hive/ql/ddl/table/ShowTablePropertiesOperation.java
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775419237
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135037/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-770965994
**[Test build #134739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134739/testReport)** for PR 31378 at commit [`e229662`](https://github.com/apache/spark/commit/e229662e390aa7770a72b88b3e7ee9fede7c1170).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568385314
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
Yea let's add an item in the migration guide.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775264375
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39620/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772420446
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39414/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771059685
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39325/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775184044
**[Test build #135032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135032/testReport)** for PR 31378 at commit [`ef6b45c`](https://github.com/apache/spark/commit/ef6b45cca32c9f405f364c942476eaa654f903cd).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568571793
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
`propertyKey` is also an internal variable name?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774630332
**[Test build #134973 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134973/testReport)** for PR 31378 at commit [`b0cf93f`](https://github.com/apache/spark/commit/b0cf93f830fa49767aa0ace69ef14e816b8080b5).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r569193648
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the property key. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when you specify the property key.
Review comment:
> nit: "whether you specify the property key." => "whether you specify the table property `key`."
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771633528
**[Test build #134772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134772/testReport)** for PR 31378 at commit [`90c07f4`](https://github.com/apache/spark/commit/90c07f4d063a09e5521f296090231bf3c1c8fad8).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771076553
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39325/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771487869
**[Test build #134772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134772/testReport)** for PR 31378 at commit [`90c07f4`](https://github.com/apache/spark/commit/90c07f4d063a09e5521f296090231bf3c1c8fad8).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r572062057
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql
##########
@@ -6,6 +6,13 @@ SHOW TBLPROPERTIES tbl;
SHOW TBLPROPERTIES tbl("p1");
SHOW TBLPROPERTIES tbl("p3");
+set spark.sql.legacy.keepCommandOutputSchema=true;
Review comment:
> yea, we can also remove changes in this file.
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567901541
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> then shall we change the schema of the v2 command?
IMO, change `ShowTablePropertiesCommand` is correct since hive's output schema is `prpt_name` and `prpt_value`, I think hive is wrong before.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772529618
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134827/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568385314
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
Yea let's add an item in the migration guide.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771651447
**[Test build #134783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134783/testReport)** for PR 31378 at commit [`8552df4`](https://github.com/apache/spark/commit/8552df4a518e67665d83d3b3578683f3140bb595).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r571982639
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -481,8 +481,14 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
throw QueryCompilationErrors.externalCatalogNotSupportShowViewsError(resolved)
}
- case ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey) =>
- ShowTablePropertiesCommand(ident.asTableIdentifier, propertyKey)
+ case s @ ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey, output) =>
+ val newOutput = if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
+ assert(output.length == 2)
+ output.tail
Review comment:
we should only do it if key is specified.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r571983429
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql
##########
@@ -6,6 +6,13 @@ SHOW TBLPROPERTIES tbl;
SHOW TBLPROPERTIES tbl("p1");
SHOW TBLPROPERTIES tbl("p3");
+set spark.sql.legacy.keepCommandOutputSchema=true;
Review comment:
Can we follow others and test the config in the scala test?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775221906
**[Test build #135037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135037/testReport)** for PR 31378 at commit [`cc9d8ac`](https://github.com/apache/spark/commit/cc9d8ac1105a336997384eab475fc82458d92b01).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775344061
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135032/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775285082
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39616/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774632174
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134973/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568415995
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> Yea let's add an item in the migration guide.
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768912618
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39177/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775116045
**[Test build #135027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135027/testReport)** for PR 31378 at commit [`728fca4`](https://github.com/apache/spark/commit/728fca4979bc7251168e8a68d85576fd97cc48fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-773380494
ping @cloud-fan Any more update on this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772298181
**[Test build #134804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134804/testReport)** for PR 31378 at commit [`348ab27`](https://github.com/apache/spark/commit/348ab27dc72d97138f17b8feb6c34e3451674434).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772228164
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39392/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r571528357
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the table property `key` or not. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only shows a `value` column when you specify the table property `key`.
Review comment:
> can we follow https://github.com/apache/spark/pull/31245/files#diff-b6c16c7a1a4e60bacda4674b52f9eecde3c8a86f281fb3ef21052267f94c1f01R44 ?
How about current?
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -490,8 +490,8 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
throw QueryCompilationErrors.externalCatalogNotSupportShowViewsError(resolved)
}
- case ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey) =>
- ShowTablePropertiesCommand(ident.asTableIdentifier, propertyKey)
+ case s @ ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey) =>
Review comment:
> and implement the legacy fallback
How about current?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774594283
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r569235871
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the table property `key`. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when you specify the table property `key`.
Review comment:
"... whether you specify the table property `key` or not."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771137858
**[Test build #134739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134739/testReport)** for PR 31378 at commit [`e229662`](https://github.com/apache/spark/commit/e229662e390aa7770a72b88b3e7ee9fede7c1170).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-770965994
**[Test build #134739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134739/testReport)** for PR 31378 at commit [`e229662`](https://github.com/apache/spark/commit/e229662e390aa7770a72b88b3e7ee9fede7c1170).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567844130
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -895,16 +895,10 @@ case class ShowTablesCommand(
* SHOW TBLPROPERTIES table_name[('propertyKey')];
* }}}
*/
-case class ShowTablePropertiesCommand(table: TableIdentifier, propertyKey: Option[String])
- extends RunnableCommand {
-
- override val output: Seq[Attribute] = {
- val schema = AttributeReference("value", StringType, nullable = false)() :: Nil
- propertyKey match {
- case None => AttributeReference("key", StringType, nullable = false)() :: schema
- case _ => schema
- }
- }
+case class ShowTablePropertiesCommand(
+ table: TableIdentifier,
+ propertyKey: Option[String],
+ override val output: Seq[Attribute]) extends RunnableCommand {
Review comment:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r569241849
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the table property `key`. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when you specify the table property `key`.
Review comment:
> "... whether you specify the table property `key` or not."
Done
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the table property `key`. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when you specify the table property `key`.
Review comment:
> "In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only shows ..."
DOne
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568352532
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> > then shall we change the schema of the v2 command?
>
> IMO, change `ShowTablePropertiesCommand` is correct since hive's output schema is `prpt_name` and `prpt_value`, I think hive is wrong before.
Also ping @wangyum @beliefer @viirya Also hope your suggestion since I believe it's a hive's bug and we don't need to follow wrong behavior in hive 1.x and 2.x
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568376839
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
Changing the output schema looks fine to me. Could you describe this change in the PR description?
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, `ShowTablePropertiesCommand`'s output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, `ShowTablePropertiesCommand`'s output only show `value` column when `propertyKey` is defined.
Review comment:
`ShowTablePropertiesCommand` is an internal class name, so please use "a `SHOW TBLPROPERTIES` clause` instead.
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, `ShowTablePropertiesCommand`'s output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, `ShowTablePropertiesCommand`'s output only show `value` column when `propertyKey` is defined.
Review comment:
"only show `value` column" -> "only show a `value` column"
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, `ShowTablePropertiesCommand`'s output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, `ShowTablePropertiesCommand`'s output only show `value` column when `propertyKey` is defined.
Review comment:
nit: "only show `value` column" -> "only show a `value` column"
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
`propertyKey` is also an internal variable name?
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
"output both show `key` and `value` columns" => "output shows `key` and `value` columns"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772414539
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39414/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r572015001
##########
File path: sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql
##########
@@ -6,6 +6,13 @@ SHOW TBLPROPERTIES tbl;
SHOW TBLPROPERTIES tbl("p1");
SHOW TBLPROPERTIES tbl("p3");
+set spark.sql.legacy.keepCommandOutputSchema=true;
Review comment:
yea, we can also remove changes in this file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r570985213
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##########
@@ -490,8 +490,8 @@ class ResolveSessionCatalog(val catalogManager: CatalogManager)
throw QueryCompilationErrors.externalCatalogNotSupportShowViewsError(resolved)
}
- case ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey) =>
- ShowTablePropertiesCommand(ident.asTableIdentifier, propertyKey)
+ case s @ ShowTableProperties(ResolvedV1TableOrViewIdentifier(ident), propertyKey) =>
Review comment:
and implement the legacy fallback
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771657971
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39366/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768924281
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39177/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r569093116
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
> `propertyKey` is also an internal variable name?
Updated to `whether you specify property key`. How about current? also ping @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768897473
**[Test build #134609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134609/testReport)** for PR 31378 at commit [`351cba7`](https://github.com/apache/spark/commit/351cba7551484406908ac00756d797cabf92aa71).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772420446
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39414/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568597437
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
> "output both show `key` and `value` columns" => "output shows `key` and `value` columns"
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771487869
**[Test build #134772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134772/testReport)** for PR 31378 at commit [`90c07f4`](https://github.com/apache/spark/commit/90c07f4d063a09e5521f296090231bf3c1c8fad8).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771550067
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39356/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772353790
> > Why are the changes needed?
> > Keep SHOW PARTITIONS command's output attribute exprId unchanged.
>
> SHOW PARTITIONS? SHOW TBLPROPERTIES?
>
> Besides, seems this PR does more in keep the commend output consistent. I think this is one of reason why the change needed.
DOne
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768929284
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39197/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769534379
**[Test build #134632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134632/testReport)** for PR 31378 at commit [`1036b2d`](https://github.com/apache/spark/commit/1036b2d732fdf173f905c117dc7f89d2c2572769).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771550067
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r570984464
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the table property `key` or not. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only shows a `value` column when you specify the table property `key`.
Review comment:
can we follow https://github.com/apache/spark/pull/31245/files#diff-b6c16c7a1a4e60bacda4674b52f9eecde3c8a86f281fb3ef21052267f94c1f01R44 ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567903240
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
Add commit `keep same as 2.x hive` as a backup and revert it. Don't be confused about this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568563894
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, `ShowTablePropertiesCommand`'s output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, `ShowTablePropertiesCommand`'s output only show `value` column when `propertyKey` is defined.
Review comment:
"only show `value` column" -> "only show a `value` column"
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, `ShowTablePropertiesCommand`'s output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, `ShowTablePropertiesCommand`'s output only show `value` column when `propertyKey` is defined.
Review comment:
nit: "only show `value` column" -> "only show a `value` column"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31378:
URL: https://github.com/apache/spark/pull/31378
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772322076
**[Test build #134820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134820/testReport)** for PR 31378 at commit [`9f850c6`](https://github.com/apache/spark/commit/9f850c6926fad397fdafa545df03b299c1a41f7f).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769614748
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134632/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775305020
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774595346
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39556/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771518537
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39356/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774633159
**[Test build #134970 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134970/testReport)** for PR 31378 at commit [`9475d66`](https://github.com/apache/spark/commit/9475d665feba1cde63a86536badb5212efe642e2).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class AlterTableSetProperties(`
* `case class AlterTableUnsetProperties(`
* ` implicit class MetadataColumnHelper(attr: Attribute) `
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774594235
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39553/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774590085
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39556/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768863296
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39177/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769023146
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134609/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775116045
**[Test build #135027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135027/testReport)** for PR 31378 at commit [`728fca4`](https://github.com/apache/spark/commit/728fca4979bc7251168e8a68d85576fd97cc48fe).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771734982
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39370/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768889150
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134590/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768889150
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134590/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768958760
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39197/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774598901
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39556/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772493396
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134820/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771889320
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134783/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774633380
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134970/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772353708
> > Change ShowTablePropertiesCommand's output schema, before when propertyKey is None will show key and value, when propertyKey not None, only show value column. Now whether propertyKey is None, will show output column value.
>
> The PR description looks not matched to the migration guide?
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771646330
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39366/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568376839
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
Changing the output schema looks fine to me. Could you describe this change in the PR description?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-770997602
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39325/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568566945
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, `ShowTablePropertiesCommand`'s output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, `ShowTablePropertiesCommand`'s output only show `value` column when `propertyKey` is defined.
Review comment:
> `ShowTablePropertiesCommand` is an internal class name, so please use "a `SHOW TBLPROPERTIES` clause` instead.
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775305020
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568572276
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
"output both show `key` and `value` columns" => "output shows `key` and `value` columns"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771149110
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134739/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r569188018
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the property key. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when you specify the property key.
Review comment:
nit: "whether you specify the property key." => "whether you specify the table property \`key\`."
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568352532
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> > then shall we change the schema of the v2 command?
>
> IMO, change `ShowTablePropertiesCommand` is correct since hive's output schema is `prpt_name` and `prpt_value`, I think hive is wrong before.
Also ping @wangyum @beliefer @viirya Also hope your suggestion since I believe it's a hive's bug and we don't need to follow wrong behavior in hive 1.x and 2.x
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> Changing the output schema looks fine to me. Could you describe this change in the PR description?
Yea, updated, Do we need to change migration guide too?
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> Yea let's add an item in the migration guide.
Done.
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, `ShowTablePropertiesCommand`'s output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, `ShowTablePropertiesCommand`'s output only show `value` column when `propertyKey` is defined.
Review comment:
> `ShowTablePropertiesCommand` is an internal class name, so please use "a `SHOW TBLPROPERTIES` clause` instead.
Done
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
> `propertyKey` is also an internal variable name?
Yea ==
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
> "output both show `key` and `value` columns" => "output shows `key` and `value` columns"
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567868743
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
> Yea we need to keep consistent. The question is which side we should follow. What's the behavior in hive?
Seems hive is same as ShowTablePropertiesCommand
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-773380494
ping @cloud-fan Any more update on this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775180757
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39610/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768924281
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39177/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775339323
**[Test build #135032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135032/testReport)** for PR 31378 at commit [`ef6b45c`](https://github.com/apache/spark/commit/ef6b45cca32c9f405f364c942476eaa654f903cd).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772357937
Looks better now. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r571522596
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output shows `key` and `value` columns whether you specify the table property `key` or not. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only shows a `value` column when you specify the table property `key`.
Review comment:
> can we follow https://github.com/apache/spark/pull/31245/files#diff-b6c16c7a1a4e60bacda4674b52f9eecde3c8a86f281fb3ef21052267f94c1f01R44 ?
updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771550067
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39356/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775288072
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39620/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r572088762
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -918,7 +912,11 @@ case class ShowTablePropertiesCommand(table: TableIdentifier, propertyKey: Optio
val propValue = catalogTable
.properties
.getOrElse(p, s"Table ${catalogTable.qualifiedName} does not have property: $p")
- Seq(Row(propValue))
+ if (conf.getConf(SQLConf.LEGACY_KEEP_COMMAND_OUTPUT_SCHEMA)) {
Review comment:
can we check `output.length` instead? to be more robust.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771649649
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134772/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774901974
ping @cloud-fan Any more need update?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769529029
FYI @cloud-fan @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771840438
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134780/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771622077
**[Test build #134780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134780/testReport)** for PR 31378 at commit [`aa348e1`](https://github.com/apache/spark/commit/aa348e19611d4d1e72ffe5978500cea9a310bb70).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775933152
ping @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771149110
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134739/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567841393
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
is it a reasonable behavior change?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771487869
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772393634
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39414/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568563430
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, `ShowTablePropertiesCommand`'s output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, `ShowTablePropertiesCommand`'s output only show `value` column when `propertyKey` is defined.
Review comment:
`ShowTablePropertiesCommand` is an internal class name, so please use "a `SHOW TBLPROPERTIES` clause` instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771649649
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134772/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775344061
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135032/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567840910
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
##########
@@ -895,16 +895,10 @@ case class ShowTablesCommand(
* SHOW TBLPROPERTIES table_name[('propertyKey')];
* }}}
*/
-case class ShowTablePropertiesCommand(table: TableIdentifier, propertyKey: Option[String])
- extends RunnableCommand {
-
- override val output: Seq[Attribute] = {
- val schema = AttributeReference("value", StringType, nullable = false)() :: Nil
- propertyKey match {
- case None => AttributeReference("key", StringType, nullable = false)() :: schema
- case _ => schema
- }
- }
+case class ShowTablePropertiesCommand(
+ table: TableIdentifier,
+ propertyKey: Option[String],
+ override val output: Seq[Attribute]) extends RunnableCommand {
Review comment:
do we need the `override val`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r568597361
##########
File path: docs/sql-migration-guide.md
##########
@@ -56,6 +56,8 @@ license: |
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
- In Spark 3.2, the usage of `count(tblName.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(tblName.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
+
+ - In Spark 3.2, a `SHOW TBLPROPERTIES` clause's output both show `key` and `value` columns whether `propertyKey` is defined. In Spark 3.1 and earlier, a `SHOW TBLPROPERTIES` clause's output only show a `value` column when `propertyKey` is defined.
Review comment:
> `propertyKey` is also an internal variable name?
Yea ==
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772358909
**[Test build #134827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134827/testReport)** for PR 31378 at commit [`6ba183e`](https://github.com/apache/spark/commit/6ba183e51216e01b599b54aeb4d4cb7bbb812ac3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-775221906
**[Test build #135037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135037/testReport)** for PR 31378 at commit [`cc9d8ac`](https://github.com/apache/spark/commit/cc9d8ac1105a336997384eab475fc82458d92b01).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771657954
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39366/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772228164
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39392/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771657971
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39366/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771622077
**[Test build #134780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134780/testReport)** for PR 31378 at commit [`aa348e1`](https://github.com/apache/spark/commit/aa348e19611d4d1e72ffe5978500cea9a310bb70).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769529029
FYI @cloud-fan @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771537248
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39356/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771851994
**[Test build #134783 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134783/testReport)** for PR 31378 at commit [`8552df4`](https://github.com/apache/spark/commit/8552df4a518e67665d83d3b3578683f3140bb595).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772311963
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134804/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-770743091
gentle ping @HyukjinKwon @cloud-fan @maropu
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772180730
**[Test build #134804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134804/testReport)** for PR 31378 at commit [`348ab27`](https://github.com/apache/spark/commit/348ab27dc72d97138f17b8feb6c34e3451674434).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768838711
**[Test build #134590 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134590/testReport)** for PR 31378 at commit [`af15746`](https://github.com/apache/spark/commit/af157469aa0138c484382278c5b03563b6b97220).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772354141
Please also update the PR title too. Currently it looks not very meaningful.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768947203
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39197/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772180730
**[Test build #134804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134804/testReport)** for PR 31378 at commit [`348ab27`](https://github.com/apache/spark/commit/348ab27dc72d97138f17b8feb6c34e3451674434).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-768958760
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31378:
URL: https://github.com/apache/spark/pull/31378#discussion_r567857623
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveCommandSuite.scala
##########
@@ -129,7 +129,7 @@ class HiveCommandSuite extends QueryTest with SQLTestUtils with TestHiveSingleto
checkAnswer(
sql(s"SHOW TBLPROPERTIES parquet_tab1('my_key1')"),
- Row("v1") :: Nil
+ Row("my_key1", "v1") :: Nil
Review comment:
Yea we need to keep consistent. The question is which side we should follow. What's the behavior in hive?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-776410205
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-774584708
**[Test build #134973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134973/testReport)** for PR 31378 at commit [`b0cf93f`](https://github.com/apache/spark/commit/b0cf93f830fa49767aa0ace69ef14e816b8080b5).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772517711
**[Test build #134827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134827/testReport)** for PR 31378 at commit [`6ba183e`](https://github.com/apache/spark/commit/6ba183e51216e01b599b54aeb4d4cb7bbb812ac3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772342663
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39407/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-771076553
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39325/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772353904
> > Does this PR introduce any user-facing change?
> > No
>
> I think this PR actually introduces user-facing change. That's why we update migration guide.
DOne
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-772311963
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134804/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of SHOW TBLPROPERTIES pass output attribute properly
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31378:
URL: https://github.com/apache/spark/pull/31378#issuecomment-769569104
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39220/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org