You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/14 06:53:53 UTC
[GitHub] [spark] beliefer opened a new pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
beliefer opened a new pull request #31178:
URL: https://github.com/apache/spark/pull/31178
### What changes were proposed in this pull request?
The current implement of `UnboundedOffsetWindowFunctionFrame` and `UnboundedPrecedingOffsetWindowFunctionFrame` only support `nth_value` that respect nulls. So nth_value will execute with `UnboundedWindowFunctionFrame` and `UnboundedPrecedingWindowFunctionFrame`.
`UnboundedWindowFunctionFrame` and `UnboundedPrecedingWindowFunctionFrame` will call `updateExpressions` of `nth_value` multiple times.
### Why are the changes needed?
Improve performance for nth_value ignore nulls over offset window
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Jenkins test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763944989
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134285/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31178:
URL: https://github.com/apache/spark/pull/31178#discussion_r557843329
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExecBase.scala
##########
@@ -149,8 +150,8 @@ trait WindowExecBase extends UnaryExecNode {
case AggregateExpression(f, _, _, _, _) => collect("AGGREGATE", frame, e, f)
case f: FrameLessOffsetWindowFunction =>
collect("FRAME_LESS_OFFSET", f.fakeFrame, e, f)
- case f: OffsetWindowFunction if !f.ignoreNulls &&
- frame.frameType == RowFrame && frame.lower == UnboundedPreceding =>
+ case f: OffsetWindowFunction if frame.frameType == RowFrame &&
+ frame.lower == UnboundedPreceding =>
frame.upper match {
Review comment:
if we go with the `case _ => collect("AGGREGATE", frame, e, f)` branch, do we still support ignore nulls?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31178:
URL: https://github.com/apache/spark/pull/31178
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31178:
URL: https://github.com/apache/spark/pull/31178#discussion_r561051697
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala
##########
@@ -305,12 +308,21 @@ class UnboundedOffsetWindowFunctionFrame(
inputIterator = input.generateIterator()
// drain the first few rows if offset is larger than one
inputIndex = 0
- while (inputIndex < offset - 1) {
- if (inputIterator.hasNext) inputIterator.next()
- inputIndex += 1
+ if (ignoreNulls) {
+ findNextRowWithNonNullInput()
+ if (nextSelectedRow == EmptyRow) {
+ // Use default values since the offset row whose input value is not null does not exist.
+ fillDefaultValue(EmptyRow)
+ } else {
+ projection(nextSelectedRow)
+ }
+ } else {
+ while (inputIndex < offset) {
+ selectedRow = WindowFunctionFrame.getNextOrNull(inputIterator)
Review comment:
In this class, `write` is empty, so `selectedRow` can be a local variable
```
var selectedRow: UnsafeRow = null
while (inputIndex < offset) {
selectedRow = WindowFunctionFrame.getNextOrNull(inputIterator)
inputIndex += 1
}
projection(selectedRow)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763653986
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38860/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760641016
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38659/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763789727
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38864/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763733954
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38864/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31178:
URL: https://github.com/apache/spark/pull/31178#discussion_r557843994
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExecBase.scala
##########
@@ -149,8 +150,8 @@ trait WindowExecBase extends UnaryExecNode {
case AggregateExpression(f, _, _, _, _) => collect("AGGREGATE", frame, e, f)
case f: FrameLessOffsetWindowFunction =>
collect("FRAME_LESS_OFFSET", f.fakeFrame, e, f)
- case f: OffsetWindowFunction if !f.ignoreNulls &&
- frame.frameType == RowFrame && frame.lower == UnboundedPreceding =>
+ case f: OffsetWindowFunction if frame.frameType == RowFrame &&
+ frame.lower == UnboundedPreceding =>
frame.upper match {
Review comment:
Nth_value support ignore nulls. lead and lag not.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-764442309
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760627504
**[Test build #134073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134073/testReport)** for PR 31178 at commit [`c5259be`](https://github.com/apache/spark/commit/c5259beea04ee20620b622f67fdb83e5bc30cd79).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760626263
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38659/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760758025
**[Test build #134073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134073/testReport)** for PR 31178 at commit [`c5259be`](https://github.com/apache/spark/commit/c5259beea04ee20620b622f67fdb83e5bc30cd79).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-764447673
@cloud-fan Thanks for your work!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763898643
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134280/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31178:
URL: https://github.com/apache/spark/pull/31178#discussion_r560988916
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala
##########
@@ -305,12 +306,22 @@ class UnboundedOffsetWindowFunctionFrame(
inputIterator = input.generateIterator()
// drain the first few rows if offset is larger than one
inputIndex = 0
- while (inputIndex < offset - 1) {
- if (inputIterator.hasNext) inputIterator.next()
- inputIndex += 1
+ if (ignoreNulls) {
+ findNextRowWithNonNullInput()
+ if (nextSelectedRow == EmptyRow) {
+ // Use default values since the offset row whose input value is not null does not exist.
+ fillDefaultValue(EmptyRow)
+ } else {
+ projection(nextSelectedRow)
+ }
+ } else {
+ while (inputIndex < offset - 1) {
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763627160
**[Test build #134274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134274/testReport)** for PR 31178 at commit [`4ba5a6b`](https://github.com/apache/spark/commit/4ba5a6b3745f6eca0aa671ae8a9c5616520533fb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763802029
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38871/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763898643
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760758882
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134073/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763700333
**[Test build #134280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134280/testReport)** for PR 31178 at commit [`64642a2`](https://github.com/apache/spark/commit/64642a22aeac80087059ef5384382ab09fda26ea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763898643
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763904422
**[Test build #134285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134285/testReport)** for PR 31178 at commit [`bf17bae`](https://github.com/apache/spark/commit/bf17bae5e8d3f7c0eb1c17d90b75618d60553d4d).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #31178:
URL: https://github.com/apache/spark/pull/31178
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763742671
**[Test build #134285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134285/testReport)** for PR 31178 at commit [`bf17bae`](https://github.com/apache/spark/commit/bf17bae5e8d3f7c0eb1c17d90b75618d60553d4d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763878043
**[Test build #134280 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134280/testReport)** for PR 31178 at commit [`64642a2`](https://github.com/apache/spark/commit/64642a22aeac80087059ef5384382ab09fda26ea).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763742671
**[Test build #134285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134285/testReport)** for PR 31178 at commit [`bf17bae`](https://github.com/apache/spark/commit/bf17bae5e8d3f7c0eb1c17d90b75618d60553d4d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31178:
URL: https://github.com/apache/spark/pull/31178#discussion_r557843994
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExecBase.scala
##########
@@ -149,8 +150,8 @@ trait WindowExecBase extends UnaryExecNode {
case AggregateExpression(f, _, _, _, _) => collect("AGGREGATE", frame, e, f)
case f: FrameLessOffsetWindowFunction =>
collect("FRAME_LESS_OFFSET", f.fakeFrame, e, f)
- case f: OffsetWindowFunction if !f.ignoreNulls &&
- frame.frameType == RowFrame && frame.lower == UnboundedPreceding =>
+ case f: OffsetWindowFunction if frame.frameType == RowFrame &&
+ frame.lower == UnboundedPreceding =>
frame.upper match {
Review comment:
Nth_value also supports ignore nulls
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763828225
**[Test build #134274 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134274/testReport)** for PR 31178 at commit [`4ba5a6b`](https://github.com/apache/spark/commit/4ba5a6b3745f6eca0aa671ae8a9c5616520533fb).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760078487
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763789727
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38864/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760078487
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38630/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760170401
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134044/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763627160
**[Test build #134274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134274/testReport)** for PR 31178 at commit [`4ba5a6b`](https://github.com/apache/spark/commit/4ba5a6b3745f6eca0aa671ae8a9c5616520533fb).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #31178:
URL: https://github.com/apache/spark/pull/31178#discussion_r561078618
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala
##########
@@ -305,12 +308,21 @@ class UnboundedOffsetWindowFunctionFrame(
inputIterator = input.generateIterator()
// drain the first few rows if offset is larger than one
inputIndex = 0
- while (inputIndex < offset - 1) {
- if (inputIterator.hasNext) inputIterator.next()
- inputIndex += 1
+ if (ignoreNulls) {
+ findNextRowWithNonNullInput()
+ if (nextSelectedRow == EmptyRow) {
+ // Use default values since the offset row whose input value is not null does not exist.
+ fillDefaultValue(EmptyRow)
+ } else {
+ projection(nextSelectedRow)
+ }
+ } else {
+ while (inputIndex < offset) {
+ selectedRow = WindowFunctionFrame.getNextOrNull(inputIterator)
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-764442309
thanks, merging to master!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760627504
**[Test build #134073 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134073/testReport)** for PR 31178 at commit [`c5259be`](https://github.com/apache/spark/commit/c5259beea04ee20620b622f67fdb83e5bc30cd79).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760758882
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134073/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #31178:
URL: https://github.com/apache/spark/pull/31178#discussion_r560962193
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala
##########
@@ -305,12 +306,22 @@ class UnboundedOffsetWindowFunctionFrame(
inputIterator = input.generateIterator()
// drain the first few rows if offset is larger than one
inputIndex = 0
- while (inputIndex < offset - 1) {
- if (inputIterator.hasNext) inputIterator.next()
- inputIndex += 1
+ if (ignoreNulls) {
+ findNextRowWithNonNullInput()
+ if (nextSelectedRow == EmptyRow) {
+ // Use default values since the offset row whose input value is not null does not exist.
+ fillDefaultValue(EmptyRow)
+ } else {
+ projection(nextSelectedRow)
+ }
+ } else {
+ while (inputIndex < offset - 1) {
Review comment:
Can we follow https://github.com/apache/spark/pull/31178/files#diff-09c876759e9b45e198238f7244eba48900058896287bf371acb27de9e25032b8R367 to write the loop?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763829858
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760641016
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38659/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763778989
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38871/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-764447673
@cloud-fan Thanks for your work!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760633826
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38659/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763699786
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38860/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763699746
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38860/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-759989714
**[Test build #134044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134044/testReport)** for PR 31178 at commit [`cdab348`](https://github.com/apache/spark/commit/cdab34831e63f8aa8c8d5d819169d0dfc1c5344c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763779295
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38864/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763904422
**[Test build #134285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134285/testReport)** for PR 31178 at commit [`bf17bae`](https://github.com/apache/spark/commit/bf17bae5e8d3f7c0eb1c17d90b75618d60553d4d).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760602651
cc @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763898643
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134280/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763742671
**[Test build #134285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134285/testReport)** for PR 31178 at commit [`bf17bae`](https://github.com/apache/spark/commit/bf17bae5e8d3f7c0eb1c17d90b75618d60553d4d).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-759989714
**[Test build #134044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134044/testReport)** for PR 31178 at commit [`cdab348`](https://github.com/apache/spark/commit/cdab34831e63f8aa8c8d5d819169d0dfc1c5344c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763700333
**[Test build #134280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134280/testReport)** for PR 31178 at commit [`64642a2`](https://github.com/apache/spark/commit/64642a22aeac80087059ef5384382ab09fda26ea).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-760160047
**[Test build #134044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134044/testReport)** for PR 31178 at commit [`cdab348`](https://github.com/apache/spark/commit/cdab34831e63f8aa8c8d5d819169d0dfc1c5344c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763829858
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763944989
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134285/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31178: [SPARK-34096][SQL] Improve performance for nth_value ignore nulls over offset window
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31178:
URL: https://github.com/apache/spark/pull/31178#issuecomment-763699786
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38860/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org