You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/10/13 08:20:35 UTC
[GitHub] [spark] beliefer opened a new pull request #30023: [SPARK-33126][SQL] Improve offset window function(Remove direction field)
beliefer opened a new pull request #30023:
URL: https://github.com/apache/spark/pull/30023
### What changes were proposed in this pull request?
The current `Lead`/`Lag` extends `OffsetWindowFunction`. `OffsetWindowFunction` contains field `direction` and use `direction` to calculates the `boundary`.
We can use single literal expression unify the two properties.
For example:
3 means `direction` is Asc and `boundary` is 3.
-3 means `direction` is Desc and `boundary` is -3.
### Why are the changes needed?
Improve the current implement of `Lead`/`Lag`.
### Does this PR introduce _any_ user-facing change?
'No'.
### How was this patch tested?
Jenkins test.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708663533
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707618645
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/34339/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-709983930
@cloud-fan Thanks for your help!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708663509
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34357/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #30023:
URL: https://github.com/apache/spark/pull/30023#discussion_r503937537
##########
File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part1.sql.out
##########
@@ -202,7 +202,7 @@ struct<ntile(3) OVER (ORDER BY ten ASC NULLS FIRST, four ASC NULLS FIRST ROWS BE
-- !query
SELECT lag(ten) OVER (PARTITION BY four ORDER BY ten), ten, four FROM tenk1 WHERE unique2 < 10
-- !query schema
-struct<lag(ten, 1, NULL) OVER (PARTITION BY four ORDER BY ten ASC NULLS FIRST ROWS BETWEEN -1 FOLLOWING AND -1 FOLLOWING):int,ten:int,four:int>
+struct<lag(ten, -1, NULL) OVER (PARTITION BY four ORDER BY ten ASC NULLS FIRST ROWS BETWEEN -1 FOLLOWING AND -1 FOLLOWING):int,ten:int,four:int>
Review comment:
The user query is `lag(ten)`. Can we override `stringArgs` in `OffsetWindowFunction` to only include `input`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-709957634
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708601575
**[Test build #129751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129751/testReport)** for PR 30023 at commit [`e73edb2`](https://github.com/apache/spark/commit/e73edb22d47e07840eba473588d6617f5b6df395).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708601575
**[Test build #129751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129751/testReport)** for PR 30023 at commit [`e73edb2`](https://github.com/apache/spark/commit/e73edb22d47e07840eba473588d6617f5b6df395).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707587309
**[Test build #129735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129735/testReport)** for PR 30023 at commit [`5ff7e23`](https://github.com/apache/spark/commit/5ff7e238d8bcc49d850779baaddf4e37acb87d7d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708663533
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #30023:
URL: https://github.com/apache/spark/pull/30023
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708778337
**[Test build #129751 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129751/testReport)** for PR 30023 at commit [`e73edb2`](https://github.com/apache/spark/commit/e73edb22d47e07840eba473588d6617f5b6df395).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707620603
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #30023:
URL: https://github.com/apache/spark/pull/30023#discussion_r504684088
##########
File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part1.sql.out
##########
@@ -202,7 +202,7 @@ struct<ntile(3) OVER (ORDER BY ten ASC NULLS FIRST, four ASC NULLS FIRST ROWS BE
-- !query
SELECT lag(ten) OVER (PARTITION BY four ORDER BY ten), ten, four FROM tenk1 WHERE unique2 < 10
-- !query schema
-struct<lag(ten, 1, NULL) OVER (PARTITION BY four ORDER BY ten ASC NULLS FIRST ROWS BETWEEN -1 FOLLOWING AND -1 FOLLOWING):int,ten:int,four:int>
+struct<lag(ten, -1, NULL) OVER (PARTITION BY four ORDER BY ten ASC NULLS FIRST ROWS BETWEEN -1 FOLLOWING AND -1 FOLLOWING):int,ten:int,four:int>
Review comment:
I don't remember the details, please check the code and find a way to change the string format here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-710050202
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-710049061
**[Test build #129889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129889/testReport)** for PR 30023 at commit [`07b3a92`](https://github.com/apache/spark/commit/07b3a926a5b3ad69b11425fef5fd131382bb1ab4).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-710050202
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #30023:
URL: https://github.com/apache/spark/pull/30023#discussion_r506204707
##########
File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part1.sql.out
##########
@@ -202,7 +202,7 @@ struct<ntile(3) OVER (ORDER BY ten ASC NULLS FIRST, four ASC NULLS FIRST ROWS BE
-- !query
SELECT lag(ten) OVER (PARTITION BY four ORDER BY ten), ten, four FROM tenk1 WHERE unique2 < 10
-- !query schema
-struct<lag(ten, 1, NULL) OVER (PARTITION BY four ORDER BY ten ASC NULLS FIRST ROWS BETWEEN -1 FOLLOWING AND -1 FOLLOWING):int,ten:int,four:int>
+struct<lag(ten, -1, NULL) OVER (PARTITION BY four ORDER BY ten ASC NULLS FIRST ROWS BETWEEN -1 FOLLOWING AND -1 FOLLOWING):int,ten:int,four:int>
Review comment:
I got it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708781156
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-709957634
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707578671
**[Test build #129733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129733/testReport)** for PR 30023 at commit [`9d6b579`](https://github.com/apache/spark/commit/9d6b579f470978aa2e5830c492ea07a01f965c55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707610958
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34341/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707620581
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34341/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707587309
**[Test build #129735 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129735/testReport)** for PR 30023 at commit [`5ff7e23`](https://github.com/apache/spark/commit/5ff7e238d8bcc49d850779baaddf4e37acb87d7d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707618634
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-709949211
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34494/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Improve offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707578671
**[Test build #129733 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129733/testReport)** for PR 30023 at commit [`9d6b579`](https://github.com/apache/spark/commit/9d6b579f470978aa2e5830c492ea07a01f965c55).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-709957614
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34494/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707722074
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707722076
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129735/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707722074
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-709929939
**[Test build #129889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129889/testReport)** for PR 30023 at commit [`07b3a92`](https://github.com/apache/spark/commit/07b3a926a5b3ad69b11425fef5fd131382bb1ab4).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707642699
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707620603
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707642708
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129733/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707618634
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707618610
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34339/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-709982581
GA passed, merging to master, thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707721325
**[Test build #129735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129735/testReport)** for PR 30023 at commit [`5ff7e23`](https://github.com/apache/spark/commit/5ff7e238d8bcc49d850779baaddf4e37acb87d7d).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707642699
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707642448
**[Test build #129733 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129733/testReport)** for PR 30023 at commit [`9d6b579`](https://github.com/apache/spark/commit/9d6b579f470978aa2e5830c492ea07a01f965c55).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class Lag(input: Expression, inputOffset: Expression, default: Expression)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #30023: [SPARK-33126][SQL] Improve offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #30023:
URL: https://github.com/apache/spark/pull/30023#discussion_r503762036
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -348,16 +348,12 @@ abstract class OffsetWindowFunction
/**
* (Foldable) expression that contains the number of rows between the current row and the row
- * where the input expression is evaluated.
+ * where the input expression is evaluated. If `offset` is a positive integer, it means that
+ * the direction of the `offset` is from front to back. If it is a negative integer, the direction
+ * of the `offset` is from back to front.
Review comment:
what if the offset is 0?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708781156
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #30023:
URL: https://github.com/apache/spark/pull/30023#discussion_r504679176
##########
File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part1.sql.out
##########
@@ -202,7 +202,7 @@ struct<ntile(3) OVER (ORDER BY ten ASC NULLS FIRST, four ASC NULLS FIRST ROWS BE
-- !query
SELECT lag(ten) OVER (PARTITION BY four ORDER BY ten), ten, four FROM tenk1 WHERE unique2 < 10
-- !query schema
-struct<lag(ten, 1, NULL) OVER (PARTITION BY four ORDER BY ten ASC NULLS FIRST ROWS BETWEEN -1 FOLLOWING AND -1 FOLLOWING):int,ten:int,four:int>
+struct<lag(ten, -1, NULL) OVER (PARTITION BY four ORDER BY ten ASC NULLS FIRST ROWS BETWEEN -1 FOLLOWING AND -1 FOLLOWING):int,ten:int,four:int>
Review comment:
You means `override def sql` ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-707604268
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34339/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-708654971
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34357/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30023:
URL: https://github.com/apache/spark/pull/30023#issuecomment-709929939
**[Test build #129889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129889/testReport)** for PR 30023 at commit [`07b3a92`](https://github.com/apache/spark/commit/07b3a926a5b3ad69b11425fef5fd131382bb1ab4).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #30023: [SPARK-33126][SQL] Simplify offset window function(Remove direction field)
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #30023:
URL: https://github.com/apache/spark/pull/30023#discussion_r503765919
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
##########
@@ -348,16 +348,12 @@ abstract class OffsetWindowFunction
/**
* (Foldable) expression that contains the number of rows between the current row and the row
- * where the input expression is evaluated.
+ * where the input expression is evaluated. If `offset` is a positive integer, it means that
+ * the direction of the `offset` is from front to back. If it is a negative integer, the direction
+ * of the `offset` is from back to front.
Review comment:
`If it is zero, it means that the offset is ignored and use current row.`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org