You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/12/11 12:52:04 UTC
[GitHub] [spark] beliefer opened a new pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
beliefer opened a new pull request #34868:
URL: https://github.com/apache/spark/pull/34868
### What changes were proposed in this pull request?
This PR used to support ANSI aggregate Function: `regr_count`
The mainstream database supports `regr_count` show below:
**Teradata**
https://docs.teradata.com/r/kmuOwjp1zEYg98JsB8fu_A/1skTm5SD0HD3Z~vii3hXAg
**Snowflake**
https://docs.snowflake.com/en/sql-reference/functions/regr_count.html#
**Oracle**
https://docs.oracle.com/en/database/oracle/oracle-database/19/sqlrf/REGR_-Linear-Regression-Functions.html#GUID-A675B68F-2A88-4843-BE2C-FCDE9C65F9A9
**H2**
http://www.h2database.com/html/functions-aggregate.html#regr_count
**Postgresql**
https://www.postgresql.org/docs/8.4/functions-aggregate.html
### Why are the changes needed?
`regr_count` is very useful.
### Does this PR introduce _any_ user-facing change?
'No'. New feature.
### How was this patch tested?
New tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996811448
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50806/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997126453
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50818/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-998037695
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50875/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996818355
**[Test build #146333 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146333/testReport)** for PR 34868 at commit [`677124b`](https://github.com/apache/spark/commit/677124bd5febefc7b291cd6231ca9cfc3aa8e107).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997702732
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50867/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-998185086
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146400/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991676586
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146093/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-998185086
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146400/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996725518
**[Test build #146333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146333/testReport)** for PR 34868 at commit [`677124b`](https://github.com/apache/spark/commit/677124bd5febefc7b291cd6231ca9cfc3aa8e107).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997673627
**[Test build #146392 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146392/testReport)** for PR 34868 at commit [`06d6790`](https://github.com/apache/spark/commit/06d679026dc989bd04c1dc6ba05cacfa3d4c1b91).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996548936
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50796/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1050564017
@cloud-fan @gengliangwang Thank you for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia edited a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034362773
@beliefer can you document the new functions in the PR description?
Basically can you list
1. the function signature (name and argument type)
2. Argument valid input range. Maybe some examples
3. Not valid argument inputs. Maybe some examples.
4. other special cases if any
There are usage comment for each expression. We can also copy documentation above into those comment. Current usage description is too vague to understand the function specification:
```
usage = """
_FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in
a group.
"""
```
Based on the function specification, we can further check if we have enough test coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034479480
@beliefer can you refer to https://issues.apache.org/jira/browse/SPARK-38063 as an example to update the function specification.
Basically instead of reading code, let's first document and discuss what the functions should be, especially on their argument type, input range, illegal/legal inputs, etc.
After we catch edges cases by walking through the specification, writing code and code review will also be easier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996578791
**[Test build #146322 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146322/testReport)** for PR 34868 at commit [`f3262db`](https://github.com/apache/spark/commit/f3262db7a257f14ead0972ba830f9ff3fe7f4744).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996818643
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146333/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] entong commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
entong commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034153454
Thanks @amaliujia . @cloud-fan can do second round review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia edited a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034362773
@beliefer can you document the new functions in the PR description?
Basically can you list
1. the function signature (name and argument type)
2. Argument valid input range. Maybe some examples
3. Not valid argument inputs. Maybe some examples.
4. other special cases if any
?
There are usage comment for each expression. We can also copy documentation above into those comment. Current usage description is too vague to understand the function specification:
```
usage = """
_FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in
a group.
"""
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia edited a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034479480
@beliefer can you refer to https://issues.apache.org/jira/browse/SPARK-38063 as an example to update the function specification?
Basically instead of reading code, let's first document and discuss what the functions should be, especially on their argument type, input range, illegal/legal inputs, etc.
After we catch edges cases by walking through the specification, writing code and code review will also be easier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #34868:
URL: https://github.com/apache/spark/pull/34868
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991675042
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50568/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991637355
**[Test build #146093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146093/testReport)** for PR 34868 at commit [`9bfd4cb`](https://github.com/apache/spark/commit/9bfd4cbd4b60064136ff4ed56982bfcbd7f886f8).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-998181755
**[Test build #146400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146400/testReport)** for PR 34868 at commit [`3423d55`](https://github.com/apache/spark/commit/3423d552897efbc4a064149e116ae99a33fd8674).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class RegrCount(left: Expression, right: Expression) extends RegressionAggregate `
* `trait RegrAvg extends RegressionAggregate `
* `case class RegrAvgX(left: Expression, right: Expression) extends RegrAvg `
* `case class RegrAvgY(left: Expression, right: Expression) extends RegrAvg `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996498344
**[Test build #146322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146322/testReport)** for PR 34868 at commit [`f3262db`](https://github.com/apache/spark/commit/f3262db7a257f14ead0972ba830f9ff3fe7f4744).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996523696
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50796/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996786262
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50806/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997935437
**[Test build #146400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146400/testReport)** for PR 34868 at commit [`3423d55`](https://github.com/apache/spark/commit/3423d552897efbc4a064149e116ae99a33fd8674).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997972023
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50875/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia edited a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034362773
@beliefer can you document the new functions in the PR description?
Basically can you list
1. the function signature (name and argument type)
2. Argument valid input range. Maybe some examples
3. Not valid argument inputs. Maybe some examples.
4. other special cases if any.
?
There are usage comment for each expression. We can also copy documentation above into those comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r814451151
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in
+ a group.
+ """,
Review comment:
If it's one line, can we use `"..."`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1050515362
The last commit just fixes the doc string and the previous commit passed all tests. I'm merging it to master, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991676462
**[Test build #146093 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146093/testReport)** for PR 34868 at commit [`9bfd4cb`](https://github.com/apache/spark/commit/9bfd4cbd4b60064136ff4ed56982bfcbd7f886f8).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class RegrAvgX(left: Expression, right: Expression) extends RegrAvg `
* `case class RegrAvgY(left: Expression, right: Expression) extends RegrAvg `
* `trait RegrAvg extends UnevaluableAggregate with ImplicitCastInputTypes with BinaryLike[Expression]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991846068
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146101/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991637355
**[Test build #146093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146093/testReport)** for PR 34868 at commit [`9bfd4cb`](https://github.com/apache/spark/commit/9bfd4cbd4b60064136ff4ed56982bfcbd7f886f8).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991831491
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50575/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991830026
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50575/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997131705
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50818/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997140501
**[Test build #146344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146344/testReport)** for PR 34868 at commit [`742e73a`](https://github.com/apache/spark/commit/742e73a21b0989d380339956851deed242b3e16e).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996753991
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50806/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997120388
**[Test build #146345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146345/testReport)** for PR 34868 at commit [`f710cdd`](https://github.com/apache/spark/commit/f710cdd4cd75831f65f0a6069cf5d58d14ca1b3c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997898604
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146392/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia edited a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034362773
@beliefer can you document the new functions in the PR description?
Basically can you list
1. the function signature (name and argument type)
2. Argument valid input range. Maybe some examples
3. Not valid argument inputs. Maybe some examples.
4. other special cases if any
There are usage comment for each expression. We can also copy documentation above into those comment. Current usage description is too vague to understand the function specification:
```
usage = """
_FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in
a group.
"""
```
Based on the function specification, we can further check if we have enough test coverage.
It's very important to have function specification. Even though there are links to other dialects' documentation. We still want our documentation to know how exactly it is same or different from others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034152854
I will take a look soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813858600
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in
+ a group.
+ """,
Review comment:
can you try `DESC FUNCTION` locally? I'm afraid multi-line string doesn't work well in function doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991675042
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991845486
**[Test build #146101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146101/testReport)** for PR 34868 at commit [`4220627`](https://github.com/apache/spark/commit/422062787708c76ed002bbcfd53523fada834b01).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997134864
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996498344
**[Test build #146322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146322/testReport)** for PR 34868 at commit [`f3262db`](https://github.com/apache/spark/commit/f3262db7a257f14ead0972ba830f9ff3fe7f4744).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997673627
**[Test build #146392 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146392/testReport)** for PR 34868 at commit [`06d6790`](https://github.com/apache/spark/commit/06d679026dc989bd04c1dc6ba05cacfa3d4c1b91).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997757839
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50867/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-998037695
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50875/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991821530
**[Test build #146101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146101/testReport)** for PR 34868 at commit [`4220627`](https://github.com/apache/spark/commit/422062787708c76ed002bbcfd53523fada834b01).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991831491
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50575/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991821530
**[Test build #146101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146101/testReport)** for PR 34868 at commit [`4220627`](https://github.com/apache/spark/commit/422062787708c76ed002bbcfd53523fada834b01).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991637355
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997119811
**[Test build #146344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146344/testReport)** for PR 34868 at commit [`742e73a`](https://github.com/apache/spark/commit/742e73a21b0989d380339956851deed242b3e16e).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997153060
**[Test build #146345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146345/testReport)** for PR 34868 at commit [`f710cdd`](https://github.com/apache/spark/commit/f710cdd4cd75831f65f0a6069cf5d58d14ca1b3c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997126722
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50819/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991637355
**[Test build #146093 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146093/testReport)** for PR 34868 at commit [`9bfd4cb`](https://github.com/apache/spark/commit/9bfd4cbd4b60064136ff4ed56982bfcbd7f886f8).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996565780
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50796/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813889501
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 2.75
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3.0
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 3.0
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+// scalastyle:on line.size.limit
+case class RegrAvgX(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression =
+ Average(If(And(IsNotNull(left), IsNotNull(right)), right, Literal.create(null, right.dataType)))
+ override def nodeName: String = "regr_avgx"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrAvgX =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
Review comment:
```suggestion
_FUNC_(expr1, expr2) - Returns the average of the dependent variable for non-null pairs in a group, where `expr1` is the dependent variable and `expr2` is the independent variable.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813888935
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
Review comment:
```suggestion
_FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group, where `expr1` is the dependent variable and `expr2` is the independent variable.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813904074
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group, where `expr1` is the dependent variable and `expr2` is the independent variable.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
Review comment:
Let's add one example or test case for checking both `regr_avgx(1, null)` and `regr_avgy(1, null)` return null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813898933
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group, where `expr1` is the dependent variable and `expr2` is the independent variable.
Review comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia edited a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034362773
@beliefer can you document the new functions in the PR description?
Basically can you list
1. the function signature (name and argument type)
2. Argument valid input range. Maybe some examples
3. Not valid argument inputs. Maybe some examples.
4. other special cases if any.
?
There are usage comment for each expression. We can also copy documentation above into those comment. Current usage description is too vague to understand the function specification:
```
usage = """
_FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in
a group.
"""
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991824862
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50575/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991675042
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991651077
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50568/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996725518
**[Test build #146333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146333/testReport)** for PR 34868 at commit [`677124b`](https://github.com/apache/spark/commit/677124bd5febefc7b291cd6231ca9cfc3aa8e107).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r772180462
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/regressions.scala
##########
@@ -0,0 +1,124 @@
+/*
Review comment:
I'd make the file name clearer: `linearRegression.scala`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997757816
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50867/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997167233
ping @MaxGekk @gengliangwang cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997898604
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146392/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997935437
**[Test build #146400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146400/testReport)** for PR 34868 at commit [`3423d55`](https://github.com/apache/spark/commit/3423d552897efbc4a064149e116ae99a33fd8674).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813894835
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 2.75
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3.0
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 3.0
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+// scalastyle:on line.size.limit
+case class RegrAvgX(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression =
+ Average(If(And(IsNotNull(left), IsNotNull(right)), right, Literal.create(null, right.dataType)))
+ override def nodeName: String = "regr_avgx"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrAvgX =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
Review comment:
> Here it should be `dependent` variable
Thank you for the found.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991846068
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146101/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996608855
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146322/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996565780
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50796/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996811448
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50806/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997154318
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146345/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997141651
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146344/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-998021400
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50875/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991676586
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146093/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034362773
@beliefer can you document the new functions in the PR description?
Basically can you list
1. the function signature (name and argument type)
2. Argument valid input range. Maybe some examples
3. Not valid argument inputs. Maybe some examples.
4. other special cases if any.
There are usage comment for each expression. We can also copy documentation above into those comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034465450
@amaliujia PR description updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r814403846
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group, where `expr1` is the dependent variable and `expr2` is the independent variable.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
Review comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813889729
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 2.75
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3.0
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 3.0
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+// scalastyle:on line.size.limit
+case class RegrAvgX(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression =
+ Average(If(And(IsNotNull(left), IsNotNull(right)), right, Literal.create(null, right.dataType)))
+ override def nodeName: String = "regr_avgx"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrAvgX =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
Review comment:
Here it should be `dependent` variable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813877841
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in
+ a group.
+ """,
Review comment:
Yes. I should use `// scalastyle:off line.size.limit`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813879505
##########
File path: sql/core/src/test/resources/sql-functions/sql-expression-schema.md
##########
@@ -68,8 +68,8 @@
| org.apache.spark.sql.catalyst.expressions.Cast | timestamp | N/A | N/A |
| org.apache.spark.sql.catalyst.expressions.Cast | tinyint | N/A | N/A |
| org.apache.spark.sql.catalyst.expressions.Cbrt | cbrt | SELECT cbrt(27.0) | struct<CBRT(27.0):double> |
-| org.apache.spark.sql.catalyst.expressions.Ceil | ceil | SELECT ceil(-0.1) | struct<CEIL(-0.1):decimal(1,0)> |
-| org.apache.spark.sql.catalyst.expressions.Ceil | ceiling | SELECT ceiling(-0.1) | struct<ceiling(-0.1):decimal(1,0)> |
+| org.apache.spark.sql.catalyst.expressions.CeilExpressionBuilder$ | ceil | SELECT ceil(-0.1) | struct<CEIL(-0.1):decimal(1,0)> |
+| org.apache.spark.sql.catalyst.expressions.CeilExpressionBuilder$ | ceiling | SELECT ceiling(-0.1) | struct<ceiling(-0.1):decimal(1,0)> |
Review comment:
Yes. The problem occurs in many of my PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813894828
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group, where `expr1` is the dependent variable and `expr2` is the independent variable.
Review comment:
On second thought, let's change (expr1, expr2) to (y, x) so that users can understanding the names of "avgx" and "avgy"
https://docs.snowflake.com/en/sql-reference/functions/regr_avgx.html
https://docs.snowflake.com/en/sql-reference/functions/regr_avgy.html
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group, where `expr1` is the dependent variable and `expr2` is the independent variable.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 2.75
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3.0
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 3.0
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+// scalastyle:on line.size.limit
+case class RegrAvgX(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression =
+ Average(If(And(IsNotNull(left), IsNotNull(right)), right, Literal.create(null, right.dataType)))
+ override def nodeName: String = "regr_avgx"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrAvgX =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the dependent variable for non-null pairs in a group, where `expr1` is the dependent variable and `expr2` is the independent variable.
Review comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813860357
##########
File path: sql/core/src/test/resources/sql-functions/sql-expression-schema.md
##########
@@ -68,8 +68,8 @@
| org.apache.spark.sql.catalyst.expressions.Cast | timestamp | N/A | N/A |
| org.apache.spark.sql.catalyst.expressions.Cast | tinyint | N/A | N/A |
| org.apache.spark.sql.catalyst.expressions.Cbrt | cbrt | SELECT cbrt(27.0) | struct<CBRT(27.0):double> |
-| org.apache.spark.sql.catalyst.expressions.Ceil | ceil | SELECT ceil(-0.1) | struct<CEIL(-0.1):decimal(1,0)> |
-| org.apache.spark.sql.catalyst.expressions.Ceil | ceiling | SELECT ceiling(-0.1) | struct<ceiling(-0.1):decimal(1,0)> |
+| org.apache.spark.sql.catalyst.expressions.CeilExpressionBuilder$ | ceil | SELECT ceil(-0.1) | struct<CEIL(-0.1):decimal(1,0)> |
+| org.apache.spark.sql.catalyst.expressions.CeilExpressionBuilder$ | ceiling | SELECT ceiling(-0.1) | struct<ceiling(-0.1):decimal(1,0)> |
Review comment:
interesting... existing tests didn't capture them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813888935
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
Review comment:
```suggestion
_FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group, where `expr1` is the independent variable and `expr2` is the dependent variable.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r813889501
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/linearRegression.scala
##########
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql.catalyst.expressions.aggregate
+
+import org.apache.spark.sql.catalyst.expressions.{And, Expression, ExpressionDescription, If, ImplicitCastInputTypes, IsNotNull, Literal, RuntimeReplaceableAggregate}
+import org.apache.spark.sql.catalyst.trees.BinaryLike
+import org.apache.spark.sql.types.{AbstractDataType, NumericType}
+
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr) - Returns the number of non-null number pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 4
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 2
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+case class RegrCount(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression = Count(Seq(left, right))
+ override def nodeName: String = "regr_count"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrCount =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
+ """,
+ examples = """
+ Examples:
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, 2), (2, 3), (2, 4) AS tab(y, x);
+ 2.75
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (2, 3), (2, 4) AS tab(y, x);
+ 3.0
+ > SELECT _FUNC_(y, x) FROM VALUES (1, 2), (2, null), (null, 3), (2, 4) AS tab(y, x);
+ 3.0
+ """,
+ group = "agg_funcs",
+ since = "3.3.0")
+// scalastyle:on line.size.limit
+case class RegrAvgX(left: Expression, right: Expression)
+ extends RuntimeReplaceableAggregate with ImplicitCastInputTypes with BinaryLike[Expression] {
+ override lazy val replacement: Expression =
+ Average(If(And(IsNotNull(left), IsNotNull(right)), right, Literal.create(null, right.dataType)))
+ override def nodeName: String = "regr_avgx"
+ override def inputTypes: Seq[AbstractDataType] = Seq(NumericType, NumericType)
+ override protected def withNewChildrenInternal(
+ newLeft: Expression, newRight: Expression): RegrAvgX =
+ this.copy(left = newLeft, right = newRight)
+}
+
+// scalastyle:off line.size.limit
+@ExpressionDescription(
+ usage = """
+ _FUNC_(expr1, expr2) - Returns the average of the independent variable for non-null pairs in a group.
Review comment:
```suggestion
_FUNC_(expr1, expr2) - Returns the average of the dependent variable for non-null pairs in a group, where `expr1` is the independent variable and `expr2` is the dependent variable.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997120388
**[Test build #146345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146345/testReport)** for PR 34868 at commit [`f710cdd`](https://github.com/apache/spark/commit/f710cdd4cd75831f65f0a6069cf5d58d14ca1b3c).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997154318
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146345/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997132410
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50819/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997134864
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997757839
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50867/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996608855
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146322/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-996818643
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146333/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997119811
**[Test build #146344 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146344/testReport)** for PR 34868 at commit [`742e73a`](https://github.com/apache/spark/commit/742e73a21b0989d380339956851deed242b3e16e).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997141651
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/146344/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-997895144
**[Test build #146392 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146392/testReport)** for PR 34868 at commit [`06d6790`](https://github.com/apache/spark/commit/06d679026dc989bd04c1dc6ba05cacfa3d4c1b91).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #34868:
URL: https://github.com/apache/spark/pull/34868#discussion_r772354743
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/regressions.scala
##########
@@ -0,0 +1,124 @@
+/*
Review comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991665780
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50568/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-991675042
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50568/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] amaliujia edited a comment on pull request #34868: [SPARK-37614][SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy
Posted by GitBox <gi...@apache.org>.
amaliujia edited a comment on pull request #34868:
URL: https://github.com/apache/spark/pull/34868#issuecomment-1034362773
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org