You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/01/05 07:09:23 UTC
[GitHub] [spark] beliefer opened a new pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
beliefer opened a new pull request #31026:
URL: https://github.com/apache/spark/pull/31026
### What changes were proposed in this pull request?
Change `FrameLessOffsetWindowFunction` as sealed abstract class so that simplify pattern match.
### Why are the changes needed?
Simplify pattern match
### Does this PR introduce _any_ user-facing change?
Yes
### How was this patch tested?
Jenkins test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755141437
**[Test build #133703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133703/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754451897
**[Test build #133660 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133660/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755149778
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133703/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755032591
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754539994
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38248/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754556484
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38248/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r551801291
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
@beliefer I think you should keep this default match. I remember Scala 2.13 issues a warning about this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552391060
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
Hm, but I thought we should keep the default match here. If `sealed` requires to remove the default match here, we shouldn't do that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552400188
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
I think that the complier misses several cases to be honest (see https://github.com/apache/spark/pull/30923/files#diff-aad995a1ccfd7ca8e2a4c7bdf5114ddda8f9017e56cf8c60bb63846b3b5f3153R48-R50 as an example).
But I got that it's easier just to trust complier for maintenance.
Either way works to me since the complier doesn't complain for now.
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
For example, we could implement some new expression that implements `FrameLessOffsetWindowFunction` later (in the same file), and we forgot to fix it here that ends up with a raw case match error. I mean "match may not be exhaustive" warning .. like we fixed it https://github.com/apache/spark/pull/30455 before.
Or does it not show the warning in this case because we're using `sealed`?
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
Hm, but I thought we should keep the default match here. If `sealed` requires to remove the default match here, we shouldn't do that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552394273
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
For example, we could implement some new expression that implements `FrameLessOffsetWindowFunction` later (in the same file), and we forgot to fix it here that ends up with a raw case match error. I mean "match may not be exhaustive" warning .. like we fixed it https://github.com/apache/spark/pull/30455 before.
Or does it not show the warning in this case because we're using `sealed`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754507883
@beliefer just out of curiosity, I think your `master` branch in your fork is not synced properly for whatever reason. When you sync your `master` branch, you should `rebase` instead of `merge`. Otherwise, the merge commits show as shown in this PR above. I think you will have to `git rebase --force` for your master branch to get rid of the merge commits above.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755135130
LGTM too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755149778
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133703/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754586924
**[Test build #133660 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133660/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754451897
**[Test build #133660 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133660/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552292021
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
@beliefer can you revert these removal here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552400188
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
I think that the complier misses several cases to be honest (see https://github.com/apache/spark/pull/30923/files#diff-aad995a1ccfd7ca8e2a4c7bdf5114ddda8f9017e56cf8c60bb63846b3b5f3153R48-R50 as an example).
But I got that it's easier just to trust complier for maintenance.
Either way works to me since the complier doesn't complain for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754566864
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38248/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552408180
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
For the example, GitHub Action job is not triggered correctly due to 3rd party action (`linter` job's `setup-r` action). Since I recovered the GitHub Action, I remember it correctly.

There is no known cases where Scala 2.13 compiler miss. If there is a missing case, we should report to Scala community.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755080859
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38291/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755048568
**[Test build #133703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133703/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754694978
**[Test build #133686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133686/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552396430
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
Scala compiler intelligently understands `sealed` semantics. That is a compilation failure in Scala 2.13.
> For example, we could implement some new expression that implements FrameLessOffsetWindowFunction later, and we forgot to fix it here.
If you see the PR description of #30455 , I wrote like the following. Since we upgrade to Scala 2.13.4, we are safe unless the committer ignores GitHub Action job.
> Fix exhaustivity issues in both Scala 2.12/2.13 (Scala 2.13.4 requires this for compilation.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552365946
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
`sealed` means we allow subclasses in the same source file. So, this is correct, @HyukjinKwon .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754766130
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38275/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755061021
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38291/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755102389
@dongjoon-hyun Thanks for your help! @HyukjinKwon Thanks for your sincerely suggestion, I will learn and try it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754566864
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38248/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552365946
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
`sealed` means we allow subclasses in the same source file. So, this is correct, @HyukjinKwon .
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
Scala compiler intelligently understands `sealed` semantics. That is a compilation failure in Scala 2.13.
> For example, we could implement some new expression that implements FrameLessOffsetWindowFunction later, and we forgot to fix it here.
If you see the PR description of #30455 , I wrote like the following. Since we upgrade to Scala 2.13.4, we are safe unless the committer ignores GitHub Action job.
> Fix exhaustivity issues in both Scala 2.12/2.13 (Scala 2.13.4 requires this for compilation.)
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
For the example, GitHub Action job is not triggered correctly due to Apache's 3rd party action ban (`linter` job's `setup-r` action). Since I recovered the GitHub Action, I remember it correctly.

There is no known cases where Scala 2.13 compiler miss. If there is a missing case, we should report to Scala community.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754594537
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133660/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754666626
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754736034
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755080859
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38291/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754735478
**[Test build #133686 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133686/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754594537
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133660/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552408180
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
For the example, GitHub Action job is not triggered correctly due to Apache's 3rd party action ban (`linter` job's `setup-r` action). Since I recovered the GitHub Action, I remember it correctly.

There is no known cases where Scala 2.13 compiler miss. If there is a missing case, we should report to Scala community.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #31026:
URL: https://github.com/apache/spark/pull/31026#discussion_r552396430
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -2150,9 +2150,6 @@ class Analyzer(override val catalogManager: CatalogManager)
lead.copy(ignoreNulls = ignoreNulls)
case lag: Lag =>
lag.copy(ignoreNulls = ignoreNulls)
- case _ =>
- throw QueryCompilationErrors.functionWithUnsupportedSyntaxError(
- owf.prettyName, "IGNORE NULLS")
Review comment:
Scala compiler intelligently understands `sealed` semantics. That is a compilation failure in Scala 2.13.
> For example, we could implement some new expression that implements FrameLessOffsetWindowFunction later
If you see the PR description of #30455 , I wrote like the following. Since we upgrade to Scala 2.13.4, we are safe unless the committer ignores GitHub Action job.
> Fix exhaustivity issues in both Scala 2.12/2.13 (Scala 2.13.4 requires this for compilation.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754736034
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/133686/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755068600
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38291/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #31026:
URL: https://github.com/apache/spark/pull/31026
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-755048568
**[Test build #133703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133703/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31026: [SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abstract class
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31026:
URL: https://github.com/apache/spark/pull/31026#issuecomment-754694978
**[Test build #133686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/133686/testReport)** for PR 31026 at commit [`bebe4c2`](https://github.com/apache/spark/commit/bebe4c21237be19d4900429b2d454e9c5ff35477).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org