You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/22 04:58:14 UTC
[GitHub] [spark] linhongliu-db opened a new pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
linhongliu-db opened a new pull request #31286:
URL: https://github.com/apache/spark/pull/31286
### What changes were proposed in this pull request?
In spark, the `count(table.*)` may cause very weird result, for example:
```
select count(*) from (select 1 as a, null as b) t;
output: 1
select count(t.*) from (select 1 as a, null as b) t;
output: 0
```
This is because spark expands `t.*` while converts `*` to count(1), this will confuse
users. After checking the ANSI standard, `count(*)` should always be `count(1)` while `count(t.*)`
is not allowed. What's more, this is also not allowed by common databases, e.g. MySQL, Oracle.
So, this PR proposes to block the ambiguous behavior and print a clear error message for users.
### Why are the changes needed?
to avoid ambiguous behavior and follow ANSI standard and other SQL engines
### Does this PR introduce _any_ user-facing change?
Yes, `count(table.*)` behavior will be blocked and output an error message.
### How was this patch tested?
newly added and existing tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564980513
##########
File path: sql/core/src/test/resources/sql-tests/inputs/count.sql
##########
@@ -19,11 +19,11 @@ SELECT
FROM testData;
-- count with multiple expressions
-SELECT count(a, b), count(b, a), count(testData.*) FROM testData;
+SELECT count(a, b), count(b, a), count(testData.a, testData.b) FROM testData;
Review comment:
sure, I changed this in both count.sql and udf-count.sql.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `count(table.*)` to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771442107
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134754/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565773596
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
+1 for the @cloud-fan advice. Also, could you leave some comment there about why we change the original test query?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765184946
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134349/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564970241
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
Yes, in pgsql
```
select * from (select 1 as a, null as b) as t1 join (select 1 as a, 'c' as b) as t2 on t1.a = t2.a
output:
a | b | a | b
-- | -- | -- | --
1 | (null) | 1 | c
select count(t1.*) from (select 1 as a, null as b) as t1 join (select 1 as a, 'c' as b) as t2 on t1.a = t2.a
output:
count
--
1
``
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770964101
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39324/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564592198
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
##########
@@ -250,6 +250,12 @@ object QueryCompilationErrors {
new AnalysisException(s"Invalid usage of '*' in $prettyName")
}
+ def tableStarInCountNotAllowedError(targetString: String): Throwable = {
Review comment:
`singleTableStar...`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771359653
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `count(table.*)` to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771442107
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134754/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771318932
**[Test build #134754 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134754/testReport)** for PR 31286 at commit [`19340d4`](https://github.com/apache/spark/commit/19340d4fadffc4efe9d4adc823b381c2ec76d590).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770655279
**[Test build #134719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134719/testReport)** for PR 31286 at commit [`1e2b307`](https://github.com/apache/spark/commit/1e2b30749039e230cefda57997a0b2dc54b0cee2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765127802
cc @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768779534
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39165/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768754897
Disallowing the syntax looks fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565519587
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
+ // be expanded while count(*) will be converted to count(1). They will produce different
+ // results and confuse users if there is any null values. For count(t1.*, t2.*), it is
Review comment:
Why `count(t1.*, t2.*)` is well-defined but `count(table.*)` is not? Isn't `count(t1.*, t2.*)` also expended like `count(table.*)` too?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770694231
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39305/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565768669
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
+ // be expanded while count(*) will be converted to count(1). They will produce different
+ // results and confuse users if there is any null values. For count(t1.*, t2.*), it is
Review comment:
Thanks @viirya for reviewing.
because expand `count(t1.*, t2.*)` is not ambiguous, I think no one will argue that `count(t1.*, t2.*)` should equal `count(1)`. Also this usage is not supported by the common database (MySQL, oracle, pgsql), so we are not in conflict with others.
but `count(table.*)` is not same. spark SQL expands the columns, pgsql converts it to `count(1)`, MySQL, Oracle (as well as ANSI) thinks this should be disallowed.
So I think block `count(table.*)` can avoid ambiguous while keeping `count(t1.*, t2.*)` can reduce the side effects.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767290861
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39059/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767324704
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134473/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771318932
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564977927
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
in spark, the second query will output 0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765181060
**[Test build #134349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134349/testReport)** for PR 31286 at commit [`e6fa9b0`](https://github.com/apache/spark/commit/e6fa9b0c313eb21147a32967bfec1f33a9dafa72).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564594013
##########
File path: sql/core/src/test/resources/sql-tests/inputs/count.sql
##########
@@ -19,11 +19,11 @@ SELECT
FROM testData;
-- count with multiple expressions
-SELECT count(a, b), count(b, a), count(testData.*) FROM testData;
+SELECT count(a, b), count(b, a), count(testData.a, testData.b) FROM testData;
Review comment:
how about `count(testData.*, testData.*)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765887802
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767356211
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770694187
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39305/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765132112
**[Test build #134349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134349/testReport)** for PR 31286 at commit [`e6fa9b0`](https://github.com/apache/spark/commit/e6fa9b0c313eb21147a32967bfec1f33a9dafa72).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765882028
**[Test build #134398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134398/testReport)** for PR 31286 at commit [`a1a5431`](https://github.com/apache/spark/commit/a1a54318538c3312cd4bb7ee45a02beddb7eb682).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768860308
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134579/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565819800
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
+ // be expanded while count(*) will be converted to count(1). They will produce different
+ // results and confuse users if there is any null values. For count(t1.*, t2.*), it is
Review comment:
Thanks @maropu for reviewing.
`select count(t.*, t.*) from values (1, null) t(a, b)` will output 0.
I'm fine with blocking `count(t1.*, t2.*)` as well but since in spark, we are allowing other similar cases that other databases don't support (and not follow ANSI), it's not harmful to keep `count(t1.*, t2.*)` as one more case. After all, introducing unnecessary behavior change (blocking `count(t1.*, t2.*)`) doesn't benefit users.
The similar usages:
count(col_a, col_b) - count multiple columns is not supported by pgsql, oracel. MySQL only support with distinct
count(struct_col.*) - expand columns in struct data type is not supported by the mentioned databases.
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
+ // be expanded while count(*) will be converted to count(1). They will produce different
+ // results and confuse users if there is any null values. For count(t1.*, t2.*), it is
Review comment:
Thanks @maropu for reviewing.
`select count(t.*, t.*) from values (1, null) t(a, b)` will output 0.
I'm fine with blocking `count(t1.*, t2.*)` as well but since in spark, we are allowing other similar cases that other databases don't support (and not follow ANSI), it's not harmful to keep `count(t1.*, t2.*)` as one more case. After all, introducing unnecessary behavior change (blocking `count(t1.*, t2.*)`) doesn't benefit users.
The similar usages:
`count(col_a, col_b)` - count multiple columns is not supported by pgsql, oracel. MySQL only support with distinct
`count(struct_col.*)` - expand columns in struct data type is not supported by the mentioned databases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767283475
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770912560
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134729/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565516787
##########
File path: docs/sql-migration-guide.md
##########
@@ -49,6 +49,8 @@ license: |
* and the method `spark.catalog.refreshTable`
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
+ - In Spark 3.2, usage of `count(table.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(table.*)` will output differently if there is any null values. To restore the behavior before spark 3.2, you can set `spark.sql.legacy.allowSingleTableStarInCount` to `true`.
Review comment:
"usage" -> "the usage"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770730367
**[Test build #134729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134729/testReport)** for PR 31286 at commit [`7e60556`](https://github.com/apache/spark/commit/7e6055657385fdaaa62e0a84c7f8c1f96b1a946a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771359653
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767319118
cc @cloud-fan, this PR is ready to review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565768669
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
+ // be expanded while count(*) will be converted to count(1). They will produce different
+ // results and confuse users if there is any null values. For count(t1.*, t2.*), it is
Review comment:
because expand `count(t1.*, t2.*)` is not ambiguous, I think no one will argue that `count(t1.*, t2.*)` should equal `count(1)`. Also this usage is not supported by the common database (MySQL, oracle, pgsql), so we are not in conflict with others.
but `count(table.*)` is not same. spark SQL expands the columns, pgsql converts it to `count(1)`, MySQL, Oracle (as well as ANSI) thinks this should be disallowed.
So I think block `count(table.*)` can avoid ambiguous while keeping `count(t1.*, t2.*)` can reduce the side effects.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765893443
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38985/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771115508
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134738/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767324704
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134473/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r567795011
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db edited a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db edited a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767319118
cc @cloud-fan, this PR is ready for review
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565768669
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
+ // be expanded while count(*) will be converted to count(1). They will produce different
+ // results and confuse users if there is any null values. For count(t1.*, t2.*), it is
Review comment:
because expand `count(t1.*, t2.*)` is not ambiguous, I think no one will argue that `count(t1.*, t2.*)` should equal `count(1)`. Also this usage is not supported by the common database (MySQL, oracle, pgsql), so we are not in conflict with others.
but `count(table.*)` is not same. spark SQL expands the columns, pgsql converts it to `count(1)`, MySQL, Oracle (as well as ANSI) thinks this should be disallowed.
So I think block `count(table.*)` can avoid ambiguous while keeping `count(t1.*, t2.*)` can lead less side effects.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765882743
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768754190
**[Test build #134579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134579/testReport)** for PR 31286 at commit [`d02e32c`](https://github.com/apache/spark/commit/d02e32c53fabd854ace4cec86deedfc6cf298f28).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768860308
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134579/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564593201
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
##########
@@ -250,6 +250,12 @@ object QueryCompilationErrors {
new AnalysisException(s"Invalid usage of '*' in $prettyName")
}
+ def tableStarInCountNotAllowedError(targetString: String): Throwable = {
+ new AnalysisException(s"It's not allowed to qualify $targetString.* " +
+ "using table name inside a count function. " +
+ "Please use '*' only or expand the table manually, e.g. court(*), count(col1, col2)")
Review comment:
how about `count($targetString.*) is now allowed. Please use ...`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768849165
**[Test build #134579 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134579/testReport)** for PR 31286 at commit [`d02e32c`](https://github.com/apache/spark/commit/d02e32c53fabd854ace4cec86deedfc6cf298f28).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765887802
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38984/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564593586
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala
##########
@@ -250,6 +250,12 @@ object QueryCompilationErrors {
new AnalysisException(s"Invalid usage of '*' in $prettyName")
}
+ def tableStarInCountNotAllowedError(targetString: String): Throwable = {
+ new AnalysisException(s"It's not allowed to qualify $targetString.* " +
+ "using table name inside a count function. " +
+ "Please use '*' only or expand the table manually, e.g. court(*), count(col1, col2)")
Review comment:
`expand the table` -> `expand the columns`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767273696
**[Test build #134473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134473/testReport)** for PR 31286 at commit [`371ae9b`](https://github.com/apache/spark/commit/371ae9b8b2e605b59c4ab8a738d88323de72ff4e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770724065
**[Test build #134719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134719/testReport)** for PR 31286 at commit [`1e2b307`](https://github.com/apache/spark/commit/1e2b30749039e230cefda57997a0b2dc54b0cee2).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770997063
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39324/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r567795360
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1559,6 +1559,14 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val ALLOW_STAR_WITH_SINGLE_TABLE_IDENTIFIER_IN_COUNT =
+ buildConf("spark.sql.legacy.allowStarWithSingleTableIdentifierInCount")
+ .internal()
+ .doc("When true, the SQL function 'count' is allowed to take single 'table.*' as parameter")
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771115508
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134738/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565776437
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1559,6 +1559,14 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val ALLOW_SINGLE_TABLE_STAR_IN_COUNT =
+ buildConf("spark.sql.legacy.allowSingleTableStarInCount")
Review comment:
How about `allowSingleTableStarInCount` -> `allowStarWithTableIdentifierInCount`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770798940
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565521077
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala
##########
@@ -155,6 +155,21 @@ class ColumnExpressionSuite extends QueryTest with SharedSparkSession {
checkAnswer(testData.as("testData").select($"testData.*"), testData.collect().toSeq)
}
+ test("star can be qualified by table name inside a non-count function") {
Review comment:
Please add JIRA number in the test name: "SPARK-34199: star can ..."
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala
##########
@@ -155,6 +155,21 @@ class ColumnExpressionSuite extends QueryTest with SharedSparkSession {
checkAnswer(testData.as("testData").select($"testData.*"), testData.collect().toSeq)
}
+ test("star can be qualified by table name inside a non-count function") {
+ checkAnswer(
+ testData.as("testData").selectExpr("hash(testData.*)"),
+ testData.as("testData").selectExpr("hash(testData.key, testData.value)")
+ )
+ }
+
+ test("star cannot be qualified by table name inside a count function") {
Review comment:
ditto.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770925348
**[Test build #134738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134738/testReport)** for PR 31286 at commit [`882ac3d`](https://github.com/apache/spark/commit/882ac3d2ba24a47b258bf368b90d38653d62db55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770944839
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39324/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767273696
**[Test build #134473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134473/testReport)** for PR 31286 at commit [`371ae9b`](https://github.com/apache/spark/commit/371ae9b8b2e605b59c4ab8a738d88323de72ff4e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770725557
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134719/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767973237
**[Test build #134530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134530/testReport)** for PR 31286 at commit [`ba80913`](https://github.com/apache/spark/commit/ba80913fc8df122f1e63aec27216f30e12884037).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768155599
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134535/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564594368
##########
File path: sql/core/src/test/resources/sql-tests/inputs/count.sql
##########
@@ -19,11 +19,11 @@ SELECT
FROM testData;
-- count with multiple expressions
-SELECT count(a, b), count(b, a), count(testData.*) FROM testData;
+SELECT count(a, b), count(b, a), count(testData.a, testData.b) FROM testData;
-- distinct count with multiple expressions
SELECT
- count(DISTINCT a, b), count(DISTINCT b, a), count(DISTINCT *), count(DISTINCT testData.*)
+ count(DISTINCT a, b), count(DISTINCT b, a), count(DISTINCT *), count(DISTINCT testData.a, testData.b)
Review comment:
ditto
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765132112
**[Test build #134349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134349/testReport)** for PR 31286 at commit [`e6fa9b0`](https://github.com/apache/spark/commit/e6fa9b0c313eb21147a32967bfec1f33a9dafa72).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770925348
**[Test build #134738 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134738/testReport)** for PR 31286 at commit [`882ac3d`](https://github.com/apache/spark/commit/882ac3d2ba24a47b258bf368b90d38653d62db55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771318932
**[Test build #134754 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134754/testReport)** for PR 31286 at commit [`19340d4`](https://github.com/apache/spark/commit/19340d4fadffc4efe9d4adc823b381c2ec76d590).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771427572
**[Test build #134754 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134754/testReport)** for PR 31286 at commit [`19340d4`](https://github.com/apache/spark/commit/19340d4fadffc4efe9d4adc823b381c2ec76d590).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771114357
**[Test build #134738 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134738/testReport)** for PR 31286 at commit [`882ac3d`](https://github.com/apache/spark/commit/882ac3d2ba24a47b258bf368b90d38653d62db55).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770758041
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39315/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771318932
**[Test build #134754 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134754/testReport)** for PR 31286 at commit [`19340d4`](https://github.com/apache/spark/commit/19340d4fadffc4efe9d4adc823b381c2ec76d590).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770694231
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39305/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765159710
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38936/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765127802
cc @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770798940
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39315/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767320955
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39067/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768020137
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r568291192
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770655279
**[Test build #134719 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134719/testReport)** for PR 31286 at commit [`1e2b307`](https://github.com/apache/spark/commit/1e2b30749039e230cefda57997a0b2dc54b0cee2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r567794818
##########
File path: docs/sql-migration-guide.md
##########
@@ -49,6 +49,8 @@ license: |
* and the method `spark.catalog.refreshTable`
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
+ - In Spark 3.2, the usage of `count(table.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(table.*)` will output differently if there is any null values. To restore the behavior before Spark 3.2, you can set `spark.sql.legacy.allowStarWithSingleTableIdentifierInCount` to `true`.
Review comment:
`table` is a keyword, so `count(table.*)` is not parse-able under ansi mode. How about `count(tblName.*)`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565514250
##########
File path: docs/sql-migration-guide.md
##########
@@ -49,6 +49,8 @@ license: |
* and the method `spark.catalog.refreshTable`
In Spark 3.1 and earlier, table refreshing leaves dependents uncached.
+ - In Spark 3.2, usage of `count(table.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(table.*)` will output differently if there is any null values. To restore the behavior before spark 3.2, you can set `spark.sql.legacy.allowSingleTableStarInCount` to `true`.
Review comment:
"before spark 3.2" -> "before Spark 3.2"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765887796
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38984/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765895116
**[Test build #134398 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134398/testReport)** for PR 31286 at commit [`a1a5431`](https://github.com/apache/spark/commit/a1a54318538c3312cd4bb7ee45a02beddb7eb682).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767339967
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39067/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565824188
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
+ // be expanded while count(*) will be converted to count(1). They will produce different
+ // results and confuse users if there is any null values. For count(t1.*, t2.*), it is
Review comment:
Let's minimize the change for now. We can open a new PR to discuss if we should forbid other cases.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768037994
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39120/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768018132
**[Test build #134530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134530/testReport)** for PR 31286 at commit [`ba80913`](https://github.com/apache/spark/commit/ba80913fc8df122f1e63aec27216f30e12884037).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767973237
**[Test build #134530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134530/testReport)** for PR 31286 at commit [`ba80913`](https://github.com/apache/spark/commit/ba80913fc8df122f1e63aec27216f30e12884037).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767273696
**[Test build #134473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134473/testReport)** for PR 31286 at commit [`371ae9b`](https://github.com/apache/spark/commit/371ae9b8b2e605b59c4ab8a738d88323de72ff4e).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767994552
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39115/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768769810
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39165/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768128221
**[Test build #134535 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134535/testReport)** for PR 31286 at commit [`30aa4db`](https://github.com/apache/spark/commit/30aa4dbe879bc28fe78baa48108128eabf943b4a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r568260449
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
nit: could you leave some comments here about why we updated this test?
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
nit: Could you leave some comments here about why we updated the tests?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768792499
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39165/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771359653
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39340/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768007726
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39115/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765148673
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768021322
**[Test build #134535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134535/testReport)** for PR 31286 at commit [`30aa4db`](https://github.com/apache/spark/commit/30aa4dbe879bc28fe78baa48108128eabf943b4a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765900147
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134398/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768792499
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39165/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565779860
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -1763,6 +1763,21 @@ class Analyzer(override val catalogManager: CatalogManager)
def expandStarExpression(expr: Expression, child: LogicalPlan): Expression = {
expr.transformUp {
case f1: UnresolvedFunction if containsStar(f1.arguments) =>
+ // SPECIAL CASE: We want to block count(table.*) because in spark, count(table.*) will
+ // be expanded while count(*) will be converted to count(1). They will produce different
+ // results and confuse users if there is any null values. For count(t1.*, t2.*), it is
Review comment:
How about the case `select count(t.*, t.*) from values (1, null) t(a, b)`? Btw, If the other databases don't support the syntax, `count(t1.*, t2.*)`, we still need to keep supporting it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770898647
**[Test build #134729 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134729/testReport)** for PR 31286 at commit [`7e60556`](https://github.com/apache/spark/commit/7e6055657385fdaaa62e0a84c7f8c1f96b1a946a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r568291192
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771348877
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39340/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767617093
makes sense to me, cc @viirya @maropu
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565873980
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
@cloud-fan , @maropu , I did more tests on pgsql, seems it is much more weird than I thought.
TLDR: in pgsql, `count(t2.*)` means how many MATCHING rows from t2 table. and spark doesn't have such a semantic. Here, if we want to keep the output unchanged. it should be `count(t2.q1, t2.q2)`
Here is the example:
```
CREATE TABLE INT8_TBL AS SELECT * FROM (VALUES (1, 2), (2, 1), (3, NULL), (NULL, NULL)) AS v(q1, q2);
select * from int8_tbl as t1 left join int8_tbl as t2 on t1.q1 = t2.q2;
t1.q1 | t1.q2 | t2.q1 | t2.q2
-- | -- | -- | --
1 | 2 | 2 | 1
2 | 1 | 1 | 2
3 | (null) | (null) | (null)
(null) | (null) | (null) | (null)
select count(t1.*), count(t2.*) from int8_tbl as t1 left join int8_tbl as t2 on t1.q1 = t2.q2;
t1.count | t2.count
-- | --
4 | 2
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565780148
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1559,6 +1559,14 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val ALLOW_SINGLE_TABLE_STAR_IN_COUNT =
+ buildConf("spark.sql.legacy.allowSingleTableStarInCount")
Review comment:
sgtm
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771359653
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39340/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565520217
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1559,6 +1559,14 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val ALLOW_SINGLE_TABLE_STAR_IN_COUNT =
+ buildConf("spark.sql.legacy.allowSingleTableStarInCount")
+ .internal()
+ .doc("When true, the SQL function 'count' is allowed to take single 'table.*' as parameter")
+ .version("3.1.1")
Review comment:
`version("3.2")`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564591118
##########
File path: docs/sql-migration-guide.md
##########
@@ -89,6 +89,8 @@ license: |
* `ALTER TABLE .. ADD PARTITION` throws `PartitionsAlreadyExistException` if new partition exists already
* `ALTER TABLE .. DROP PARTITION` throws `NoSuchPartitionsException` for not existing partitions
+ - In Spark 3.1, usage of `count(table.*)` is blocked to avoid producing ambiguous results. Because `count(*)` and `count(table.*)` will output differently if there is any null values. To restore the behavior before spark 3.1, you can set `spark.sql.legacy.allowSingleTableStarInCount` to `true`.
Review comment:
We are in 3.2 now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768020137
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565777014
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -1559,6 +1559,14 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val ALLOW_SINGLE_TABLE_STAR_IN_COUNT =
+ buildConf("spark.sql.legacy.allowSingleTableStarInCount")
Review comment:
allowStarWithSingleTableIdentifierInCount? Should we make this more clear?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771440146
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770725557
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134719/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765885993
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38984/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770674748
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39305/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765893633
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134399/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768041107
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39120/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565032049
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
since it's pgsql test, shall we use `count(1)` to match pgsql test intention?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767321168
**[Test build #134473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134473/testReport)** for PR 31286 at commit [`371ae9b`](https://github.com/apache/spark/commit/371ae9b8b2e605b59c4ab8a738d88323de72ff4e).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31286:
URL: https://github.com/apache/spark/pull/31286
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765882743
**[Test build #134399 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134399/testReport)** for PR 31286 at commit [`283cd2f`](https://github.com/apache/spark/commit/283cd2f0dd7a32ea32f041ac17768be1f8e0b648).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770730367
**[Test build #134729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134729/testReport)** for PR 31286 at commit [`7e60556`](https://github.com/apache/spark/commit/7e6055657385fdaaa62e0a84c7f8c1f96b1a946a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770776569
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39315/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r568260449
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
nit: could you leave some comments here about why we updated this test?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r568261101
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
nit: Could you leave some comments here about why we updated the tests?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768036417
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39120/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765132112
**[Test build #134349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134349/testReport)** for PR 31286 at commit [`e6fa9b0`](https://github.com/apache/spark/commit/e6fa9b0c313eb21147a32967bfec1f33a9dafa72).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] linhongliu-db commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
linhongliu-db commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r565873980
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
@cloud-fan , @maropu , I did more tests on pgsql, seems it is much more wired than I thought.
TLDR: in pgsql, `count(t2.*)` means how many MATCHING rows from t2 table. and spark doesn't have such a semantic. Here, if we want to keep the output unchanged. it should be `count(t2.q1, t2.q2)`
Here is the example:
```
CREATE TABLE INT8_TBL AS SELECT * FROM (VALUES (1, 2), (2, 1), (3, NULL), (NULL, NULL)) AS v(q1, q2);
select * from int8_tbl as t1 left join int8_tbl as t2 on t1.q1 = t2.q2;
t1.q1 | t1.q2 | t2.q1 | t2.q2
-- | -- | -- | --
1 | 2 | 2 | 1
2 | 1 | 1 | 2
3 | (null) | (null) | (null)
(null) | (null) | (null) | (null)
select count(t1.*), count(t2.*) from int8_tbl as t1 left join int8_tbl as t2 on t1.q1 = t2.q2;
t1.count | t2.count
-- | --
4 | 2
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765893596
**[Test build #134399 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134399/testReport)** for PR 31286 at commit [`283cd2f`](https://github.com/apache/spark/commit/283cd2f0dd7a32ea32f041ac17768be1f8e0b648).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767290861
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39059/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767308149
**[Test build #134480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134480/testReport)** for PR 31286 at commit [`fc4d3dd`](https://github.com/apache/spark/commit/fc4d3dd246005c0fdb2a2320eb7d5f29b2fb89b2).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768021322
**[Test build #134535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134535/testReport)** for PR 31286 at commit [`30aa4db`](https://github.com/apache/spark/commit/30aa4dbe879bc28fe78baa48108128eabf943b4a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768754190
**[Test build #134579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134579/testReport)** for PR 31286 at commit [`d02e32c`](https://github.com/apache/spark/commit/d02e32c53fabd854ace4cec86deedfc6cf298f28).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31286:
URL: https://github.com/apache/spark/pull/31286
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765184946
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134349/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-770997063
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39324/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768155599
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134535/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-765159710
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38936/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31286:
URL: https://github.com/apache/spark/pull/31286#discussion_r564596061
##########
File path: sql/core/src/test/resources/sql-tests/inputs/postgreSQL/join.sql
##########
@@ -743,20 +743,20 @@ select * from a left join b on i = x and i = y and x = i;
--
-- test NULL behavior of whole-row Vars, per bug #5025
--
-select t1.q2, count(t2.*)
+select t1.q2, count(t2.q1, t2.q2)
Review comment:
`count(t2.*)` means `count(1)` in pgsql?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771440146
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-768041107
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39120/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-767356211
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31286:
URL: https://github.com/apache/spark/pull/31286#issuecomment-771338055
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39340/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org