You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/13 09:38:20 UTC

[GitHub] [spark] AngersZhuuuu opened a new pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

AngersZhuuuu opened a new pull request #32149:
URL: https://github.com/apache/spark/pull/32149


   ### What changes were proposed in this pull request?
   According to https://github.com/apache/spark/pull/29087#discussion_r612267050,  add UT in `transform.sql`
   
   It seems that distinct is not recognized as a reserved word here
   
   ```
   +-- !query
   SELECT TRANSFORM(distinct b, a, c)
     USING 'cat' AS (a, b, c)
   FROM script_trans
   WHERE a <= 4
   -- !query schema
   struct<>
   -- !query output
   org.apache.spark.sql.AnalysisException
   cannot resolve 'distinct' given input columns: [script_trans.a, script_trans.b, script_trans.c]; line 1 pos 17
   
   
   -- !query
   explain extended SELECT TRANSFORM(distinct b, a, c)
                      USING 'cat' AS (a, b, c)
                    FROM script_trans
                    WHERE a <= 4
   -- !query schema
   struct<plan:string>
   -- !query output
   == Parsed Logical Plan ==
   'ScriptTransformation [*], cat, [a#x, b#x, c#x], ScriptInputOutputSchema(List(),List(),None,None,List(),List(),None,None,false)
   +- 'Project ['distinct AS b#x, 'a, 'c]
      +- 'Filter ('a <= 4)
         +- 'UnresolvedRelation [script_trans], [], false
   
   == Analyzed Logical Plan ==
   org.apache.spark.sql.AnalysisException: cannot resolve 'distinct' given input columns: [script_trans.a, script_trans.b, script_trans.c]; line 1 pos 34;
   'ScriptTransformation [*], cat, [a#x, b#x, c#x], ScriptInputOutputSchema(List(),List(),None,None,List(),List(),None,None,false)
   +- 'Project ['distinct AS b#x, a#x, c#x]
      +- Filter (a#x <= 4)
         +- SubqueryAlias script_trans
            +- View (`script_trans`, [a#x,b#x,c#x])
               +- Project [cast(a#x as int) AS a#x, cast(b#x as int) AS b#x, cast(c#x as int) AS c#x]
                  +- Project [a#x, b#x, c#x]
                     +- SubqueryAlias script_trans
                        +- LocalRelation [a#x, b#x, c#x]
   ```
   
   
   ### Why are the changes needed?
   
   
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   
   ### How was this patch tested?
   Added Ut
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r612565963



##########
File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out
##########
@@ -638,3 +639,15 @@ struct<b:string>
 1
 2
 3
+
+
+-- !query
+SELECT TRANSFORM(distinct b, a, c)

Review comment:
       BTW why do we support `TRANSFORM(a as b)`? is it from Hive? Normal function parameters can't have alias.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r612894887



##########
File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out
##########
@@ -638,3 +639,15 @@ struct<b:string>
 1
 2
 3
+
+
+-- !query
+SELECT TRANSFORM(distinct b, a, c)

Review comment:
       > BTW why do we support `TRANSFORM(a as b)`? is it from Hive? Normal function parameters can't have alias.
   
   Hive not support this. 
   
   ![image](https://user-images.githubusercontent.com/46485123/114645556-4a7ff400-9d0c-11eb-91eb-bc679ea0039a.png)
   
   may be we should not use 
   ```
   namedExpressionSeq
       : namedExpression (',' namedExpression)*
       ;
   
   (SELECT kind=TRANSFORM '(' namedExpressionSeq ')'
               | kind=MAP namedExpressionSeq
               | kind=REDUCE namedExpressionSeq)
   ```
   
   maybe 
   ```
   expressionSeq 
       : expression(',' expression) 
       ;
   
   (SELECT kind=TRANSFORM '(' expressionSeq ')'
               | kind=MAP expressionSeq
               | kind=REDUCE eExpressionSeq)
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r612370686



##########
File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out
##########
@@ -638,3 +639,15 @@ struct<b:string>
 1
 2
 3
+
+
+-- !query
+SELECT TRANSFORM(distinct b, a, c)

Review comment:
       ah the parser treats it as `distinct as b`.
   
   Can we improve the parser to recognize `distinct` correctly and give a clear error message?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818664860


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41869/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818604897


   FYI @cloud-fan 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818600199


   **[Test build #137289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137289/testReport)** for PR 32149 at commit [`118f242`](https://github.com/apache/spark/commit/118f242e60cae053777719d557023d57c87f967e).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819417507


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41919/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613061073



##########
File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out
##########
@@ -638,3 +639,15 @@ struct<b:string>
 1
 2
 3
+
+
+-- !query
+SELECT TRANSFORM(distinct b, a, c)

Review comment:
       let's have 2 PRs. forbid distinct and forbid alias are two things.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819415997






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #32149: [SPARK-31936][SQL][FOLLOWUP] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613070928



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
##########
@@ -88,6 +88,11 @@ object QueryParsingErrors {
     new ParseException("DISTRIBUTE BY is not supported", ctx)
   }
 
+  def transformNotSupportQuantifierError(ctx: ParserRuleContext): Throwable = {
+    new ParseException("TRANSFORM not support quantifier DISTINCT/ALL " +

Review comment:
       `quantifier DISTINCT/ALL` -> `DISTINCT/ALL`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819417507


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41919/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819589708


   thanks, merging to master!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818785153


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137289/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819382791


   **[Test build #137340 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137340/testReport)** for PR 32149 at commit [`542aa46`](https://github.com/apache/spark/commit/542aa461394a2cd1b6fa44d51d0c7def5b2a2beb).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #32149: [SPARK-31936][SQL][FOLLOWUP] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613064953



##########
File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out
##########
@@ -638,3 +639,15 @@ struct<b:string>
 1
 2
 3
+
+
+-- !query
+SELECT TRANSFORM(distinct b, a, c)

Review comment:
       > let's have 2 PRs. forbid distinct and forbid alias are two things.
   
   I have make a new ticket for this one. and changed the title.
   
   How about current




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819568688


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137340/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613056798



##########
File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out
##########
@@ -638,3 +639,15 @@ struct<b:string>
 1
 2
 3
+
+
+-- !query
+SELECT TRANSFORM(distinct b, a, c)

Review comment:
       > yea, let's fix it as well.
   
   Should I fix both issue in this pr or  in separated PR?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818664860


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41869/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818600199


   **[Test build #137289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137289/testReport)** for PR 32149 at commit [`118f242`](https://github.com/apache/spark/commit/118f242e60cae053777719d557023d57c87f967e).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819568688


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137340/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818785153


   
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137289/
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819382791


   **[Test build #137340 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137340/testReport)** for PR 32149 at commit [`542aa46`](https://github.com/apache/spark/commit/542aa461394a2cd1b6fa44d51d0c7def5b2a2beb).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818651807






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613052655



##########
File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out
##########
@@ -638,3 +639,15 @@ struct<b:string>
 1
 2
 3
+
+
+-- !query
+SELECT TRANSFORM(distinct b, a, c)

Review comment:
       yea, let's fix it as well.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613052655



##########
File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out
##########
@@ -638,3 +639,15 @@ struct<b:string>
 1
 2
 3
+
+
+-- !query
+SELECT TRANSFORM(distinct b, a, c)

Review comment:
       yea, let's fix it.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-819562442


   **[Test build #137340 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137340/testReport)** for PR 32149 at commit [`542aa46`](https://github.com/apache/spark/commit/542aa461394a2cd1b6fa44d51d0c7def5b2a2beb).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #32149: [SPARK-31936][SQL][FOLLOWUP] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
AngersZhuuuu commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613073123



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
##########
@@ -88,6 +88,11 @@ object QueryParsingErrors {
     new ParseException("DISTRIBUTE BY is not supported", ctx)
   }
 
+  def transformNotSupportQuantifierError(ctx: ParserRuleContext): Throwable = {
+    new ParseException("TRANSFORM not support quantifier DISTINCT/ALL " +

Review comment:
       Done

##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
##########
@@ -88,6 +88,11 @@ object QueryParsingErrors {
     new ParseException("DISTRIBUTE BY is not supported", ctx)
   }
 
+  def transformNotSupportQuantifierError(ctx: ParserRuleContext): Throwable = {
+    new ParseException("TRANSFORM not support quantifier DISTINCT/ALL " +
+      "for input expression seq", ctx)

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #32149: [SPARK-31936][SQL][FOLLOWUP] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613069968



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
##########
@@ -88,6 +88,11 @@ object QueryParsingErrors {
     new ParseException("DISTRIBUTE BY is not supported", ctx)
   }
 
+  def transformNotSupportQuantifierError(ctx: ParserRuleContext): Throwable = {
+    new ParseException("TRANSFORM not support quantifier DISTINCT/ALL " +

Review comment:
       not support -> does not support




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan commented on a change in pull request #32149: [SPARK-31936][SQL][FOLLOWUP] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #32149:
URL: https://github.com/apache/spark/pull/32149#discussion_r613070583



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
##########
@@ -88,6 +88,11 @@ object QueryParsingErrors {
     new ParseException("DISTRIBUTE BY is not supported", ctx)
   }
 
+  def transformNotSupportQuantifierError(ctx: ParserRuleContext): Throwable = {
+    new ParseException("TRANSFORM not support quantifier DISTINCT/ALL " +
+      "for input expression seq", ctx)

Review comment:
       `for input expression seq` -> `in inputs`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #32149: [SPARK-28227][SQL][FOLLOWUP] Add test case about transform expression with `distinct`

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32149:
URL: https://github.com/apache/spark/pull/32149#issuecomment-818779400


   **[Test build #137289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137289/testReport)** for PR 32149 at commit [`118f242`](https://github.com/apache/spark/commit/118f242e60cae053777719d557023d57c87f967e).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] cloud-fan closed pull request #32149: [SPARK-35069][SQL] TRANSFORM forbidden `DISTICNT` and `ALL`, also make the error clear

Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #32149:
URL: https://github.com/apache/spark/pull/32149


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org