You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/09/28 02:53:37 UTC
[GitHub] [spark] beliefer opened a new pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
beliefer opened a new pull request #29886:
URL: https://github.com/apache/spark/pull/29886
### What changes were proposed in this pull request?
https://github.com/apache/spark/pull/29604 supports the ANSI SQL NTH_VALUE.
We should override the `prettyName` and `sql`.
### Why are the changes needed?
Make the name of nth_value correct.
To show the ignoreNulls parameter correctly.
### Does this PR introduce _any_ user-facing change?
'No'.
### How was this patch tested?
Jenkins test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495685180
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
Review comment:
nit: `NthValue` -> `Nth value` in the doc
Can we also add an example about `ignore nulls` while we're here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699955392
**[Test build #129170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129170/testReport)** for PR 29886 at commit [`fe9b8ab`](https://github.com/apache/spark/commit/fe9b8ab88f1b515110d80cc0223111f4fcbf254a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699820425
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699756406
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699847535
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33785/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495710029
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
override lazy val evaluateExpression: AttributeReference = result
- override def toString: String = s"$prettyName($input, $offset)${if (ignoreNulls) " ignore nulls"}"
+ override def prettyName: String = "nth_value"
+ override def sql: String =
+ s"$prettyName(${input.sql}, $offset)${if (ignoreNulls) " ignore nulls"}"
Review comment:
Thank you! @HyukjinKwon
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r496348770
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
Review comment:
The current `ignore nulls` is very ustomized. I think it needs to be redesigned to be more common to all window functions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699823885
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33781/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699826476
**[Test build #129170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129170/testReport)** for PR 29886 at commit [`fe9b8ab`](https://github.com/apache/spark/commit/fe9b8ab88f1b515110d80cc0223111f4fcbf254a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699842944
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33782/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495697154
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
override lazy val evaluateExpression: AttributeReference = result
- override def toString: String = s"$prettyName($input, $offset)${if (ignoreNulls) " ignore nulls"}"
+ override def prettyName: String = "nth_value"
+ override def sql: String =
+ s"$prettyName(${input.sql}, $offset)${if (ignoreNulls) " ignore nulls"}"
Review comment:
```suggestion
s"$prettyName(${input.sql}, ${offsetExpr.sql})" + (if (ignoreNulls) " ignore nulls" else "")
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #29886:
URL: https://github.com/apache/spark/pull/29886
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699820425
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699860245
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33785/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699842975
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699811873
**[Test build #129167 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129167/testReport)** for PR 29886 at commit [`fe9b8ab`](https://github.com/apache/spark/commit/fe9b8ab88f1b515110d80cc0223111f4fcbf254a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495677916
##########
File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part3.sql.out
##########
@@ -391,7 +391,7 @@ SELECT nth_value(four, 0) OVER (ORDER BY ten), ten, four FROM tenk1
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
-cannot resolve 'nthvalue(spark_catalog.default.tenk1.`four`, 0)' due to data type mismatch: The 'offset' argument of nth_value must be greater than zero but it is 0.; line 1 pos 7
+cannot resolve 'nth_value(spark_catalog.default.tenk1.`four`, 0)()' due to data type mismatch: The 'offset' argument of nth_value must be greater than zero but it is 0.; line 1 pos 7
Review comment:
Do you know why we have an extra `()`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699833101
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699860279
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699801623
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495697154
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
override lazy val evaluateExpression: AttributeReference = result
- override def toString: String = s"$prettyName($input, $offset)${if (ignoreNulls) " ignore nulls"}"
+ override def prettyName: String = "nth_value"
+ override def sql: String =
+ s"$prettyName(${input.sql}, $offset)${if (ignoreNulls) " ignore nulls"}"
Review comment:
```suggestion
s"$prettyName(${input.sql}, $offset)" + (if (ignoreNulls) " ignore nulls" else "")
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699820297
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699957160
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699811873
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699741396
**[Test build #129163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129163/testReport)** for PR 29886 at commit [`7f3c033`](https://github.com/apache/spark/commit/7f3c0338b8a3c92ab7adff6ea512e5bbad93714b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699756390
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33778/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495701315
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
Review comment:
Unfortunately there is no SQL API for it yet. It's only available in DataFrame API. We will add examples when we add the SQL API.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699957160
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699802095
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r496349005
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
Review comment:
ahh gotya
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699842975
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495701581
##########
File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part3.sql.out
##########
@@ -391,7 +391,7 @@ SELECT nth_value(four, 0) OVER (ORDER BY ten), ten, four FROM tenk1
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
-cannot resolve 'nthvalue(spark_catalog.default.tenk1.`four`, 0)' due to data type mismatch: The 'offset' argument of nth_value must be greater than zero but it is 0.; line 1 pos 7
+cannot resolve 'nth_value(spark_catalog.default.tenk1.`four`, 0)()' due to data type mismatch: The 'offset' argument of nth_value must be greater than zero but it is 0.; line 1 pos 7
Review comment:
Can we fix it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699860287
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33785/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699820466
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129166/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699860279
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699833065
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33781/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699741396
**[Test build #129163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129163/testReport)** for PR 29886 at commit [`7f3c033`](https://github.com/apache/spark/commit/7f3c0338b8a3c92ab7adff6ea512e5bbad93714b).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r496349619
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
Review comment:
I mean the `ignore nulls` feature.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699826476
**[Test build #129170 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129170/testReport)** for PR 29886 at commit [`fe9b8ab`](https://github.com/apache/spark/commit/fe9b8ab88f1b515110d80cc0223111f4fcbf254a).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699802095
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r496334515
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -669,7 +669,9 @@ case class NthValue(input: Expression, offsetExpr: Expression, ignoreNulls: Bool
Review comment:
@cloud-fan, isn't it registered in SQL side? I see the example added in the original PR https://github.com/apache/spark/commit/8b09536cdf5c5477114cc11601c8b68c70408279#diff-5e6cbbb39e10b12ccd9d6a0bc3de31a4R153
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699834724
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33782/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699802110
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129163/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699824516
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495681967
##########
File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part3.sql.out
##########
@@ -391,7 +391,7 @@ SELECT nth_value(four, 0) OVER (ORDER BY ten), ten, four FROM tenk1
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
-cannot resolve 'nthvalue(spark_catalog.default.tenk1.`four`, 0)' due to data type mismatch: The 'offset' argument of nth_value must be greater than zero but it is 0.; line 1 pos 7
+cannot resolve 'nth_value(spark_catalog.default.tenk1.`four`, 0)()' due to data type mismatch: The 'offset' argument of nth_value must be greater than zero but it is 0.; line 1 pos 7
Review comment:
Because the code in `Expression`
```
def sql: String = {
val childrenSQL = children.map(_.sql).mkString(", ")
s"$prettyName($childrenSQL)"
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-700385059
@HyukjinKwon @cloud-fan Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699756406
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-700360738
Merged to master.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #29886:
URL: https://github.com/apache/spark/pull/29886#discussion_r495703279
##########
File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part3.sql.out
##########
@@ -391,7 +391,7 @@ SELECT nth_value(four, 0) OVER (ORDER BY ten), ten, four FROM tenk1
struct<>
-- !query output
org.apache.spark.sql.AnalysisException
-cannot resolve 'nthvalue(spark_catalog.default.tenk1.`four`, 0)' due to data type mismatch: The 'offset' argument of nth_value must be greater than zero but it is 0.; line 1 pos 7
+cannot resolve 'nth_value(spark_catalog.default.tenk1.`four`, 0)()' due to data type mismatch: The 'offset' argument of nth_value must be greater than zero but it is 0.; line 1 pos 7
Review comment:
https://github.com/apache/spark/pull/29886#discussion_r495697154 fixes it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699751308
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33778/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #29886: [SPARK-27951][SQL][FOLLOWUP] Improve the window function nth_value
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #29886:
URL: https://github.com/apache/spark/pull/29886#issuecomment-699833101
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org